Tuesday, 18 December 2012

Database Design For Google App Engine - Tutorial

Are you looking for app engine database designing tutorials in the Internet? Well unfortunately there is very few information about it out there. Most of them are generic in nature, praising the stale common features of cloud computing we've heard over and over again - scalability, fast, non-relational stuffs in general etc. I kept searching for hours to find that most of the results were not useful to make a real world applications in GAE, and finally stumbled on some very valuable sources and techniques from several sources and gathered a final concepts you need to know to model data in Google app engine. I'll share it very comprehensively in this tutorial so that you don't need to gather it from different places as I did.

I'm quite experienced in relational database ( Oracle and MySQL) previously. I've been trying to design a product in Google App Engine since they provide free Java hosting. It was then that I got stuck wondering how to design my tables and schema in the non-relational database provided by the app engine. Eventually, this is how I learned it.

Before Designing Database in Google App Engine

The Google App Engine uses 'BigTable' as the database, it's a non-relational database with high scale-ability and storage capacity. So learning about some basic concepts of the non-relational database before hand will be very helpful.
Concepts to learn: 

  • Basic difference between relational and non-relational database
  • What is indexing ? 

Creating Tables (Entities) in Google App Engine

Actually, there are no tables in gae. Everything is stored as Entity. There are three ways you can structure your backend.

  • With the low-level datastore api
  • JPA 
  • JDO

Setting relations in Google App Engine

Although gae uses a non-relational database, you can still structure relations if you use JPA or JDO in you backend. This a widely used technique

Querying the Google App Engine Datastore

This is a very important portion. There are two important things that you need to learn about gae bigtable that will simplify the way you store data for your queries.

  • Multivalued list properties
  • Self- merge

Both these concepts are explained with example in the Brett Saltkin's video at Google I/O. The examples are shown in python. If you are a java developer you can see some excellent examples in the book Google App Engine Java and GWT Application Development by Daniel Guermeur and Amy Unruh. This book provides a live example of playing with the Google Datastore and contains all you would ever need to create a highly scalable web application in Google App Engine. If you don't want GWT just ignore that part, the book is worth the money for the Datastore part alone. 

These are the two videos by Brett Slatkin (Google) that explain the app engine datastore in much detail and help you get started. I watched these two videos and read this book. That's it. You're ready to run fast with java on cloud! 

Udacity Course On Web Development in Google App Engine

To learn about scale-ability there is an exclusive free course in Udacity here just on the topic. It's taught by the founder of reddit.com and he clearly explains the whole scaling things happening in reddit, one of the sites with the highest number of visitors. He shows the entire course demo implementation in gae (and that was a jackpot for me!). They offer the entire course videos free to download here . I've been toiling hard with app engine before I got these resources. So I thought sharing this might help other who are stepping the foot in waters.

Each post takes two days or more of research and three hours of typing to bring out the high quality you see here. If you admire my effort, consider giving +1 to this post and Subscribe me in Facebook.  You can also Follow Me on Twitter to get all the new stuffs on GAE. 

Get My Next Post In Your Inbox