google appengine的数据存储API

众所周知,google数据存储使用的是BigTable,而非传统意义上的数据库,Google AppEngine也不例外,但为了方便程序员使用,Google提供了数据存储API,来使得大家可以传统意义上的SQL来使用Google的BigTable分布式数据存储系统。

  1. from google.appengine.ext import db
  2. from google.appengine.api import users
  3. #数据库模型
  4. class Pet(db.Model):
  5. name = db.StringProperty(required=True)
  6. type = db.StringProperty(required=True, choices=set(["cat", "dog", "bird"]))
  7. birthdate = db.DateProperty()
  8. weight_in_pounds = db.IntegerProperty()
  9. spayed_or_neutered = db.BooleanProperty()
  10. owner = db.UserProperty()
  11. #创建数据对象
  12. pet = Pet(name="Fluffy",
  13. type="cat",
  14. owner=users.get_current_user())
  15. pet.weight_in_pounds = 24
  16. #将数据提交到数据存储系统
  17. pet.put()
具体的,Google准备了2套查询接口:GQL和查询对象接口,例如
  1. if users.get_current_user():
  2. user_pets = db.GqlQuery("SELECT * FROM Pet WHERE pet.owner = :1",
  3. users.get_current_user())
  4. for pet in user_pets:
  5. pet.spayed_or_neutered = True
  6. db.put(user_pets)
一 数据实体
google AppEngine中每个数据对象被称为实体,每个实体具有几个属性。
每个实体有一个唯一的关键词,最简单的关键词是数字ID,ID也可以是应用所指定的字符串。
应用可以通过关键词,或根据对象的属性来提取数据。
模型接口
应用通过模型来定义所需要使用的数据,模型都需要定义为db.Model类的子类。模型定义了数据实体以及对应的属性,每个属性都是Property类的子类,每个属性接受一些参数用来定义该属性是否必须赋值,或者定义默认值。例如:
  1. from google.appengine.ext import db
  2. class Pet(db.Model):
  3. name = db.StringProperty(required=True)
  4. type = db.StringProperty(required=True, choices=set(["cat", "dog", "bird"]))
  5. birthdate = db.DateProperty()
  6. weight_in_pounds = db.IntegerProperty()
  7. spayed_or_neutered = db.BooleanProperty()
  8. owner = db.UserProperty(required=True)
三 扩展模型接口
基于python的动态性,google app engine提供了不同于传统数据库的功能,该功能基于expando类,使用该类可以动态设置属性,例如:
  1. class Person(db.Expando):
  2. first_name = db.StringProperty()
  3. last_name = db.StringProperty()
  4. hobbies = db.StringListProperty()
  5. p = Person(first_name="Albert", last_name="Johnson")
  6. p.hobbies = ["chess", "travel"]
  7. p.chess_elo_rating = 1350
  8. p.travel_countries_visited = ["Spain", "Italy", "USA", "Brazil"]
  9. p.travel_trip_count = 13
p是Person的实例,但p可以对一些Person类中没有定义的属性进行赋值。和固定属性不同,对于未定义动态属性的实例而言,对应的属性是不存在的,而不是None(比较绕口,不知道大家明白不明白)。在查询的时候,数据存储系统很智能,只返回定义了动态属性的实例,例如:
  1. p1 = Person()
  2. p1.favorite = 42
  3. p1.put()
  4. p2 = Person()
  5. p2.favorite = "blue"
  6. p2.put()
  7. p3 = Person()
  8. p3.put()
  9. people = db.GqlQuery("SELECT * FROM Person WHERE favorite < :1", 50)
  10. # people has p1, but not p2 or p3
  11. people = db.GqlQuery("SELECT * FROM Person WHERE favorite > :1", 50)
  12. # people has no results
属性与类型
字符串、长字符串、二进制数据
appengine支持Unicode字符串,因此不需要为中文问题而担忧。
  1. class MyModel(db.Model):
  2. string = db.StringProperty()
  3. obj = MyModel()
  4. # Python Unicode literal syntax fully describes characters in a text string.
  5. obj.string = u"kittens"
  6. # unicode() converts a byte string to a Unicode value using the named codec.
  7. obj.string = unicode("kittens", "latin-1")
  8. # A byte string is assumed to be text encoded as ASCII (the 'ascii' codec).
  9. obj.string = "kittens"
  10. # Short string properties can be used in query filters.
  11. results = db.GqlQuery("SELECT * FROM MyModel WHERE string = :1", u"kittens")
长字符串例子:
  1. class MyModel(db.Model):
  2. text = db.TextProperty()
  3. obj = MyModel()
  4. # Text() can take a Unicode value.
  5. obj.text = db.Text(u"lots of kittens")
  6. # Text() can take a byte string and the name of an encoding.
  7. obj.text = db.Text("lots of kittens", "latin-1")
  8. # If no encoding is specified, a byte string is assumed to be ASCII text.
  9. obj.text = db.Text("lots of kittens")
  10. # Text properties can store large values.
  11. obj.text = db.Text(open("a_tale_of_two_cities.txt").read(), "utf-8")
二进制数据例子:
  1. class MyModel(db.Model):
  2. blob = db.BlobProperty()
  3. obj = MyModel()
  4. obj.blob = db.Blob(open("image.png").read())
注意:长字符串和二进制数据是不被索引的,因此对该字段的查询是不会成功的。
列表:
  1. class MyModel(db.Model):
  2. numbers = db.ListProperty(long)
  3. obj = MyModel()
  4. obj.numbers = [2, 4, 6, 8, 10]
  5. obj.numbers = ["hello"]# ERROR: MyModel.numbers must be a list of longs.
  6. # Get all entities where numbers contains a 6.
  7. results = db.GqlQuery("SELECT * FROM MyModel WHERE numbers = 6")
  8. # Get all entities where numbers contains at least one element less than 10.
  9. results = db.GqlQuery("SELECT * FROM MyModel WHERE numbers < 10")
引用(类似于表间关联):
  1. class FirstModel(db.Model):
  2. prop = db.IntegerProperty()
  3. class SecondModel(db.Model):
  4. reference = db.ReferenceProperty(FirstModel)
  5. obj1 = FirstModel()
  6. obj1.prop = 42
  7. obj1.put()
  8. obj2 = SecondModel()
  9. # A reference value is the key of another entity.
  10. obj2.reference = obj1.key()
  11. # Assigning a model instance to a property uses the entity's key as the value.
  12. obj2.reference = obj1
  13. obj2.put()
注意:长字符串、二进制数据不会被索引

创建、更新记录(想了半天,还是觉得用记录还描述比较贴切,实体...太奇怪了)
  1. pet = Pet(name="Fluffy",
  2. type="cat",
  3. owner=users.get_current_user())
  4. pet.put()
  5. db.put(pet)
  6. if users.get_current_user():
  7. user_pets = db.GqlQuery("SELECT * FROM Pet WHERE pet.owner = :1",
  8. users.get_current_user())
  9. for pet in user_pets:
  10. pet.spayed_or_neutered = True
  11. db.put(user_pets)
还是很容易了,直接将对应的记录做一个put操作就可以创建、或者更新记录了。
查询记录集
对象查询:
  1. class Story(db.Model):
  2. title = db.StringProperty()
  3. date = db.DateTimeProperty()
  4. query = Story.all()
  5. query.filter('title =', 'Foo')
  6. query.order('-date')
  7. query.ancestor(key)#这句没明白做什么的?先留着吧。
  8. # These methods can be chained together on one line.
  9. query.filter('title =', 'Foo').order('-date').ancestor(key)
GQL查询:
  1. # Parameters can be bound with positional arguments.
  2. query = db.GqlQuery("SELECT * FROM Story WHERE title = :1 "
  3. "AND ANCESTOR IS :2 "
  4. "ORDER BY date DESC",
  5. 'Foo', key)
  6. # Or, parameters can be bound with keyword arguments.
  7. query = db.GqlQuery("SELECT * FROM Story WHERE title = :title "
  8. "AND ANCESTOR IS :parent "
  9. "ORDER BY date DESC",
  10. title='Foo', parent=key)
  11. # String, number and Boolean values can be literal values in the string.
  12. query = db.GqlQuery("SELECT * FROM Story WHERE title = 'Foo' "
  13. "AND ANCESTOR IS :parent "
  14. "ORDER BY date DESC",
  15. parent=key)
  16. query = Story.gql("WHERE title = :title "
    "AND ANCESTOR IS :parent "
    "ORDER BY date DESC",
    title='Foo', parent=key)

获取数据
  1. results = query.fetch(10)
  2. for result in results:
  3. print "Title: " + result.title
说真的,这个方式获取数据,比使用Python的DB API还简单呢,不过还有其他方式。
  1. for result in query:
  2. print "Title: " + result.title
可以很容易地发现,对query直接进行遍历可以不需要设定查询数据的条数。
注意:Google AppEngine中每个查询最多返回1000条数据,不管limit和offset设置的值为多少。例如,对于大于1000条的数据,如果offset设置为100,那么实际返回900条数据。 我想知道的是,如果有1万条数据,那么每次返回100条,不知道是否可以将1万条数据全部遍历,如果不可以的话,那么岂不是非常不妙啊?
要获取一条数据也很简单,如下:
  1. entity.put()
  2. key = entity.key()
  3. # ...
  4. entity = db.get(key)
  5. pets = GqlQuery("SELECT * FROM Pet WHERE name = :1", "Fluffy")
  6. pet = pets.get()
  7. owner_name = pet.owner.name
  8. #在模板中如下使用
  9. obj = MyModel(name="Foo")
  10. self.response.write('%s' % (str(obj.key()),
  11. obj.name()))
  12. # ...
  13. #在页面中如下使用
  14. key_name = self.request.get('key')
  15. obj = db.get(db.Key(key_name))
删除数据

q = db.GqlQuery("SELECT * FROM Message WHERE create_date < :1", earliest_date)results = q.fetch(10)for result in results:result.delete()
# or...
q = db.GqlQuery("SELECT * FROM Message WHERE create_date < :1", earliest_date)results = q.fetch(10)db.delete(results)
九 ID Or Key_name
如果没有设置ID的话,可以使用obj.key()获得某条记录的ID
可以使用db.Key(ID)或者Entity.Key(ID)获取obj。

索引的配置 Index.yaml 例子:
  1. indexes:
  2. - kind: Cat
  3. ancestor: no
  4. properties:
  5. - name: name
  6. - name: age
  7. direction: desc
  8. - kind: Cat
  9. properties:
  10. - name: name
  11. direction: asc
  12. - name: whiskers
  13. direction: desc
  14. - kind: Store
  15. ancestor: yes
  16. properties:
  17. - name: business
  18. direction: asc
  19. - name: owner
  20. direction: asc
ancestor如果有源查询或者Query,为True,默认为False
注意:实际上,如果你没有对该文件进行配置,google appengine会对它进行默认的配置,当然是为了优化查询了。

附录:
Model Class:(文档:http://code.google.com/intl/zh-CN/appengine/docs/datastore/modelclass.html)
类方法:
  • Model.get()
  • Model.get_by_id()
  • Model.get_by_key_name()
  • Model.get_or_insert()
  • Model.all()
  • Model.gql()
  • Model.kind()
  • Model.properties()
实例方法:
  • key()
  • put()
  • delete()
  • is_saved()
  • parent()
  • parent_key()
  • to_xml()
Query类
实例方法:
  • filter()
  • order()
  • ancestor()
  • get()
  • fetch()
  • count()
支持的属性——属性类型

Property class Value type Sort order
StringProperty str
unicode
Unicode (str is treated as ASCII)
BooleanProperty bool False < True
IntegerProperty int
long
Numeric
FloatProperty float Numeric
DateTimeProperty
DateProperty
TimeProperty
datetime.datetime Chronological
ListProperty
StringListProperty
list of a supported type If ascending, by least element; if descending, by greatest element
ReferenceProperty
SelfReferenceProperty
db.Key By path elements (kind, ID or name, kind, ID or name...)
UserProperty users.User By email address (Unicode)
BlobProperty db.Blob (not orderable)
TextProperty db.Text (not orderable)
CategoryProperty db.Category Unicode
LinkProperty db.Link Unicode
EmailProperty db.Email Unicode
GeoPtProperty db.GeoPt By latitude, then longitude
IMProperty db.IM Unicode
PhoneNumberProperty db.PhoneNumber Unicode
PostalAddressProperty db.PostalAddress Unicode
RatingProperty db.Rating Numeric
异常
exception Error() exception BadArgumentError() exception BadFilterError() exception BadKeyError() exception BadPropertyError() exception BadQueryError() exception BadRequestError() exception BadValueError() exception ConfigurationError() exception DuplicatePropertyError() exception InternalError() exception KindError() exception NotSavedError() exception PropertyError() exception ReservedWordError() exception Rollback() exception Timeout() exception TransactionFailedError() exception CapabilityDisabledError()五 事务处理
http://code.google.com/intl/zh-CN/appengine/docs/datastore/transactions.html



    推荐阅读