python|python -gevent协程发http请求

  • 【python|python -gevent协程发http请求】为什么要用协程,python在执行并发任务时有:
  • 多线程用锁,很容易造成数据不稳定,无法使用发挥多核CPU的能力
  • 多进程耗资源
  • Python通过yield提供了对协程的基本支持,但是不完全。而第三方的gevent为Python提供了比较完善的协程支持。
    • 安装gevent
    • 安装: pip install greenlet
#-*- coding: utf-8 -*- import urllib.request # url = "http://r.qzone.qq.com/cgi-bin/user/cgi_personal_card?uin=284772894" url = "http://gc.ditu.aliyun.com/geocoding?a="+urllib.request.quote("苏州市") HEADER = {'Accept':'text/html,application/xhtml+xml,application/xml; q=0.9,*/*; q=0.8','User-Agent':'Mozilla/5.0 (Windows NT 6.1; rv:29.0) Gecko/20100101 Firefox/29.0'}# 自定义请求的方法,get post def f(url): import urllib import urllib.request data=https://www.it610.com/article/urllib.request.urlopen(url).read() z_data=data.decode('UTF-8') print(z_data)# 我自己封装了gevent 的方法,重载了run from gevent import Greenlet class MyGreenlet(Greenlet): def __init__(self, func): Greenlet.__init__(self) self.func = func def _run(self): # gevent.sleep(self.n) self.funccount = 3 green_let = [] for i in range(0, count): green_let.append(MyGreenlet(f(url))) for j in range(0, count): green_let[j].start() for k in range(0, count): green_let[k].join()

  • 运行结果
D:\app\Python34\python.exe D:/app/dgm/field_study.py {"lon":120.58531,"level":2,"address":"","cityName":"","alevel":4,"lat":31.29888} {"lon":120.58531,"level":2,"address":"","cityName":"","alevel":4,"lat":31.29888} {"lon":120.58531,"level":2,"address":"","cityName":"","alevel":4,"lat":31.29888}

  • 更多参考代码
  • 线程,进程,CPU,内存,硬盘
  • python-协程yield

    推荐阅读