InfoSpace Tech Blog

InfoSpace AWS Case Study

| | Comments

Amazon Web Services released a detailed case study based on our recent migration of InfoSpace’s search application to AWS. Key takeaways from the case study include:

  • Multi-region cloud presence
  • Microsoft .Net stack
  • 6 month migration – during which traffic increased by 30%
  • 20% improvement in international response times (10% domestic)
  • Reduced capital budget by 32%

You can read all about it here. We are planning to write a series on this blog detailing our cloud migration, so stay tuned.

Feedly OAuth2 Authentication Workflow in Android

| | Comments

Androidtm supports the ability for users to pass OAuth2 credentials stored on their device straight to Googletm Services (as documented here: http://developer.android.com/google/play-services/auth.html). But let’s say you want to authenticate against a non-Google service such as Feedly (http://developer.feedly.com) using your Google account credentials.

Google does not permit the OAuth2 Access Tokens stored on your device to be sent to anyone besides a Google Service. From a security standpoint this is exactly what the user wants, otherwise a malicious app could hijack the users Google OAuth2 token. We want to avoid situations where the user entering Google credentials on a non-Google controlled environment.

So what is the appropriate workflow for a service which wishes to utilize Google purely as an authentication mechanism? Feedly looks something like this:

  • User makes login request to Feedly
  • Feedly responds with an authentication url – this is controlled and generated by Google
  • User browses to authentication url
  • User submits credentials
  • Google validates credentials
    • If valid, Google notifies Feedly and redirects User to a new URL with a special short lived code generated by Feedly
    • If not valid, user can attempt authentication again
  • Once validated, Feedly Local Client retrieves code from URL
  • Feedly Local Client uses code against Service to obtain a refresh token and access token

At this point the user is validated against the Google OAuth2 credentials. The client app consuming the 3rd party service (feedly) can be reasonably certain the person logged in should have access to the login information. Because of this, the client app should be permitted to make API calls against the 3rd party service call utilizing the access token. Again, since the access token was generated based on a successful OAuth2 authentication against the user’s Google account, we can be reasonably confident that any requests made with the access token can be made on behalf of the previously authenticated OAuth2 account.

On a web/mobile web app the login process will look fairly seamless. Basically a series of quick redirects with some pauses in the workflow for user login input.

On a native Android application, an HTTP request running asynchronously in the background cannot be used from start to finish since at some point the user must enter credentials in the Google Authentication URL webpage. To do this we must embed a WebView within the application.

We recently went through the exercise of putting together a proof-of-concept Android application which leveraged the Feedly API. While doing so we noticed the Authentication piece is fairly boilerplate code and follows the workflow of many other OAuth2-exposed APIs.

If you wish to view our implementation or use it in your Android application as a Library project the code is accessible here:

https://github.com/infospace/android_oauth2_webview

As we want to provide a high level of device coverage, we used the compatibility library fragments. There is even a framework in place to create calls to the various Feedly API methods. See com.infospace.feedly.requests.RetrieveSubscriptionsRequests.java in the github repo for reference if you wish to look into creating more Feedly API requests.

Simply import the Android library project to integrate it with your existing Fragment-based Android project. When you wish to display the authentication portion, all you have to do is initialize some helper classes on activity start, and then push the fragment. Once the user authenticates the fragment automatically saves the refresh/access tokens for further use within the application.

If you have any questions please leave a comment!

Android and Google are trademarks of Google Inc.

Building for Multiple Screens

| | Comments

I recently presented a talk on Building For Multiple Screens at the Big Android BBQ with the goal of bridging the gap between the concepts and theory behind building for multiple screens and practice of building for multiple screens. The talk was received well and I thought it would be helpful to turn it into a blog post.

There are three main things that, if you pay attention to, will help you build your app so that it scales and feels natural on a variety of screen sizes.

  • Planning and understanding your application workflow and navigation.
  • Creating layouts for your workflow that allow you to take full advantage of multiple screen sizes.
  • Implement your application workflow using and taking advantage of the idioms built in to AndroidTM.

This post isn’t going to go into any depth on the first two, but if you’re interested I’ve posted my presentation as well as a sample RSS Reader application which will help walk you through the concepts.

The Workflow

One of the most common workflows I see people asking for help with is the multi-pane workflow. A typical example of this workflow is when you start with a list, tap on a row in the list, and get additional content related to the row you initially clicked on.

On a phone we’d expect tapping on the row to navigate us to another screen with the additional data. On a tablet, where we have a lot more room for content, we have the expectation of being able to keep the context of our original list while also viewing the additional data about the row we selected.

Working in a Polyglot Environment

| | Comments

Websters Dictionary defines Polyglot as a mixture or confusion of languages or nomenclatures. Mixture..confusion…. As computer scientists we’re taught to not co-mingle or mix languages, platforms, metaphors and to write code that avoids confusion. When applying the term polyglot to computer science and taking the definition at face value it sounds just a tad bit negative or like something you want to avoid. It’s hard enough to learn one language/platform well, right? Doesn’t adding more languages into the mix just muddy the waters? In this post I want to give some practical reasons why a mixture of languages/platforms can help you be a better computer scientist and a more productive developer.

You may already work in a polyglot environment and not even be aware of it.

Are you a web developer? Do you write server side code hosted by Apache, Nginx, or IIS? Do you work on mobile applications? Do you make any 3rd party API calls?

If you answered yes to any of those questions you’re most likely already working in a polyglot environment, at least from a language perspective. A typical web development environment, at a minimum, uses HTML, Javascript, and CSS. Calling an API? You’re most likely using XML or JSON as well. Maybe you’re using ASP.NET MVC or Ruby on Rails. All of those scenarios require a polyglot environment from a language perspective. Frameworks like ASP.NET and Rails provide mechanisms to blur the lines of the languages that are used and the frameworks that use them, but you’re polyglot nonetheless.

What can we learn from a polyglot environment?

Guard Your Java Code With AutoTest Tooling

| | Comments

I recently moved into doing Android development after spending a few years in Ruby and Node.js (after many, many years in .NET). I have worked hard to shorten my personal code-health feedback loop to be as real-time as possible, so I can fix problems while retaining flow. While many folks work on a set of changes and tests and run those tests every-so-often, I found it to be extremely helpful to get the feedback automatically every time I save changes to a file.

I have not found a suitable free solution for this type of thing for Java. The solutions I have found typically require Eclipse or IntelliJ as well, and I’ve been more productive sticking with Vim and Bash lately. That being said, I wanted a solution that could support both IDE users and Text Editor users, just like I’ve had in Ruby.

InfoSpace is happy to announce that we have released our Guard::Java project to the community, for free under an MIT-style license!

Guard::Java

Guard::Java is built on top of my favorite auto-testing framework for Ruby, called Guard, and provides automatic, background testing of your code as you are working, without you having to stop your workflow! It provides this capability through the following features:

Storing User Passwords Using Salt + Hashing

| | Comments

Many developers store passwords wrong. Are you one of them? Likely, you’ve stored them in a vulnerable way in the past and maybe you still are. If your company doesn’t have a standard pattern for storing user passwords, or if it does and the pattern doesn’t contain the words “Salt” and “Hash”, this blog post is for you.

When writing user passwords to a datastore, never store them in plain text. If your datastore’s security is compromised, an attacker can easily steal user login credentials, unleashing a major PR incident on your watch. Don’t worry, it isn’t much effort to plan ahead and only incur a minor PR incident, none of which is your fault.

Is there a better way to store passwords than plain text?

Yes! Instead of storing the actual password text, store a hash of that password. Use a strong hashing algorithm like SHA256 or SHA512 to hash the password text and store the result of that. When a user needs to be authenticated, hash the submitted password and compare the two hashes. If they are the same, the password is correct. Unfortunately, hashing alone is not all that much more secure due to something called a rainbow table. Rainbow tables are very large sets of hashed passwords that can be used to convert hashes back to plain text passwords. These tables are readily available through BitTorrent.

Is there a better way to store passwords than using hashes?

Yes! Instead of storing the hash of the password, store a hash of a salted password. A salt is a random string (and unique to each password in your system) that you will append to the password before hashing it. And get this, you’ll store that salt in your datastore in plain text in the column right next to the hash!

What? Are you out of your mind?!

Route53 DNS Failover Webinar

| | Comments

This morning, Amazon Web Services hosted a webinar on using Route 53 DNS failover to provide high-availability solutions. Sean Meckley, Product Manager for Route 53, shared how this feature allows you to build applications that are resilient to failures. We were invited to share how InfoSpace uses Route 53 to enable automatic failover between AWS regions. I shared some details on our multi-region architecture, how we setup latency based routing with health checks, how we tested the service to make sure it worked for our use cases, and our results (we get automatic failover between AWS regions in about 150 seconds).

You can get the slides here and watch the presentation here (we talk about InfoSpace starting around the 25:00 mark).

To the Cloud!

| | Comments

We’ve just finished moving all of InfoSpace’s search traffic to the cloud. It’s been a monumental effort and we’ve learned a lot along the way. As of today, 100% of our search traffic is served up via Amazon Web Services (AWS).

Our journey began a little less than a year ago. We spent the first few months prototyping our architecture in AWS. During these months, we used this time to prove that our services could run in the cloud, evaluated cloud providers, and did some initial performance testing.

The real work started in earnest just after the start of 2013. In the past five months, we’ve organically developed a skilled team of engineers who are now well versed in all-things AWS: from VPCs, ELBs and ASGs to load-balanced DNS, Asgard and cloud security, we’ve built up a lot of skills and experience in a very short time.

Simultaneously with our move to AWS, we have had a project to consolidate our two datacenters down to one smaller presence. There’s a ton of logistical difficulties, contractural commitments and resource challenges we worked out – managing the timing and numerous dependencies was critical to the success of this project.

We’re going to cover several topics over the next few months about our plan to move out of the datacenters to the cloud, specific technologies and architecture choices we made, as well as how things are running as we gain more operational experience with it. Come back and visit our blog often as we chronicle our journey.

Upcoming topics

  • Application architecture for multi-region deployments
  • Global load-balancing using Route 53
  • Fire and Forget: A performance testing approach
  • Our Cloud-Ops approach
  • Our resilient, redundant AWS architecture
  • Lessons learned, landmines to avoid

Meanwhile, our Director of IT/Operations, Wayson, will start a series discussing our datacenter consolidation project.

We’re looking forward to sharing our experiences with you. And, if you’re looking for a great place to work with cloud technologies, InfoSpace is hiring!

What It’s Like to Work at InfoSpace: My First Two Months

| | Comments

Back in the beginning of April, I joined the ranks of InfoSpace. As you would expect, I was filled with the typical excitement and anticipation of someone who is about to start working for a new company. What will I be working on? What will the team be like? What’s the culture like? And of course, how fast will my work machine be?

Having been around the block in this industry, I have to say I was really surprised when I began working at InfoSpace. It really is unique!

Let me explain by giving you a peek inside at my experience over the last two months. If you’re someone who believes software engineering is as much engineering as it is art, then you’ll be right at home here. InfoSpace places a great deal of emphasis on the craftsmanship of writing code. So much so, they have several book clubs that read and then meet to discuss chapters from the book Clean Code, by Robert C. Martin. At the meetings, the groups get together to discuss what they’ve read. They debate the relevance of the points in the chapter, or share experiences about how what they read helped them improve their craft. Along these lines, InfoSpace provides a clearly stated definition of what is means to be ‘done’ with a given task. Included in that definition are incorporating unit tests for the code check-in and having a code review. Reviewers take time to offer suggestions of ways to make the code more elegant or more efficient. To put it another way, the focus is not just on getting it done, but getting it done right, even if that means it takes a little bit longer. Ultimately, everyone at InfoSpace improves as an engineer.

Side Projects and the Intrapreneurial Spirit

| | Comments

Every engineer should have a project on the side. An initiative focused on improving the business, controlled completely by the engineer. My side projects get me up in the morning and get me excited about coming to work. That’s not to say I don’t enjoy my normal work activities, but my side projects are green field, controlled 100% by me and are exciting.

These types of projects are the reason you got into computer science in the first place. Who comes up with the feature requirements? You do. Who defines the development process? You do. Who gets the glory when the project is the next big thing? You get the idea.

Engineer driven projects are becoming increasingly important to keep businesses relevant and evolving. They keep products fresh, technology fresher, and can help make a lot of money. You should be a leader in your company to push this movement forward. You should be an intrapreneur.

Where do I get ideas for projects?

The first criteria for picking a side project is that you are excited about it. No one is asking you to get this work done so if you’re not exciting about blazing the trail with your ideas and passion, the project isn’t going to go anywhere. You should feel a deep sense of ownership that its fate is in your hands. No one is going check up on it and see how it is going. Own it. Get it done.