湖南工业大学网站|湖南工业大学电费数据抓取(完美校园版)

前言 之前写了一篇我们学校电费查询数据获取的博客,那篇文章的数据源是工大助手,因为工大助手对参数进行了一些加密,所以要获取到数据必须要进行破解,这样难度就增加了很多,今天我要爬的是完美校园网站的,这个软件非常开放,就是经常无法访问,首先这个网站所有有关查询电费的接口都是GET请求,其次所有请求都没有身份检查,所以这次的爬虫程序非常简单,就按顺序访问几个接口就可以轻松拿到数据了。我把这个功能已经集成到我的小程序和安卓应用中去了
湖南工业大学网站|湖南工业大学电费数据抓取(完美校园版)
文章图片
湖南工业大学网站|湖南工业大学电费数据抓取(完美校园版)
文章图片

准备 首先将手机和电脑放到同一个局域网,然后打开电脑端的fiddler工具,然后将手机的代理服务器设置成电脑的ip , 端口写8888 , 设置好以后用手机访问 http://电脑ip:8888 , 里面有两个链接,其中一个是下载证书的链接 , 将证书下载下来导入手机 , 现在手机上所有http协议和https协议的数据包都会经过电脑了 , fiddler就可以实现对手机抓包了
开始 首先打完美校园充电费的界面,点击请选择房间,这个时候到fiddler中去看就会返现发送了好几个请求

http://h5cloud.17wanxiao.com:8080//CloudPayment/user/getRoom.do?payProId=1567&schoolcode=786&optype=1&areaid=0&buildid=0&unitid=0&levelid=0&businesstype=2

结果
{"code":"SUCCESS","msg":"获取房间信息成功","roomlist":[{"id":"1016","name":"东校区","factorycode":null},{"id":"4","name":"西校区","factorycode":null}]}

其中这个请求是下面要用到的,这个请求是获取校区代号的
然后在完美校园中任意选择一个校区(我这里选择河西校区) , 继续去找数据包
http://h5cloud.17wanxiao.com:8080/CloudPayment/user/getRoom.do?payProId=1567&schoolcode=786&optype=2&areaid=4&buildid=0&unitid=0&levelid=0&businesstype=2

结果
{"code":"SUCCESS","msg":"获取房间信息成功","roomlist":[{"id":"618","name":"1栋1层商业门面","factorycode":null},{"id":"598","name":"21栋宾馆及门面电表","factorycode":null},{"id":"604","name":"22栋1层住户及门面","factorycode":null},{"id":"612","name":"23栋1层住户及门面","factorycode":null},{"id":"620","name":"2栋1层商业门面","factorycode":null},{"id":"622","name":"2食堂商业门面","factorycode":null},{"id":"20","name":"第五食堂门面","factorycode":null},{"id":"986","name":"青年公寓","factorycode":null},{"id":"400","name":"学生公寓10栋","factorycode":null},{"id":"416","name":"学生公寓11栋","factorycode":null},{"id":"464","name":"学生公寓12栋","factorycode":null},{"id":"508","name":"学生公寓13栋","factorycode":null},{"id":"528","name":"学生公寓14栋","factorycode":null},{"id":"542","name":"学生公寓15栋","factorycode":null},{"id":"414","name":"学生公寓16栋","factorycode":null},{"id":"436","name":"学生公寓17栋","factorycode":null},{"id":"456","name":"学生公寓18栋","factorycode":null},{"id":"478","name":"学生公寓19栋","factorycode":null},{"id":"274","name":"学生公寓1栋","factorycode":null},{"id":"496","name":"学生公寓20栋","factorycode":null},{"id":"170","name":"学生公寓21栋","factorycode":null},{"id":"172","name":"学生公寓22栋","factorycode":null},{"id":"174","name":"学生公寓23栋","factorycode":null},{"id":"100","name":"学生公寓24栋","factorycode":null},{"id":"114","name":"学生公寓25栋","factorycode":null},{"id":"128","name":"学生公寓26栋","factorycode":null},{"id":"176","name":"学生公寓27栋","factorycode":null},{"id":"130","name":"学生公寓28栋","factorycode":null},{"id":"132","name":"学生公寓29栋","factorycode":null},{"id":"288","name":"学生公寓2栋","factorycode":null},{"id":"232","name":"学生公寓30栋","factorycode":null},{"id":"246","name":"学生公寓31栋","factorycode":null},{"id":"260","name":"学生公寓32栋","factorycode":null},{"id":"556","name":"学生公寓33栋","factorycode":null},{"id":"570","name":"学生公寓34栋","factorycode":null},{"id":"584","name":"学生公寓35栋","factorycode":null},{"id":"302","name":"学生公寓3栋","factorycode":null},{"id":"316","name":"学生公寓4栋","factorycode":null},{"id":"330","name":"学生公寓5栋","factorycode":null},{"id":"344","name":"学生公寓6栋","factorycode":null},{"id":"358","name":"学生公寓7栋","factorycode":null},{"id":"372","name":"学生公寓8栋","factorycode":null},{"id":"386","name":"学生公寓9栋","factorycode":null},{"id":"1122","name":"学生宿舍36栋","factorycode":null},{"id":"1124","name":"学生宿舍37栋","factorycode":null},{"id":"1126","name":"学生宿舍38栋","factorycode":null}]}

这个接口他的作用是获取楼栋信息,其中接口中areaid参数表示上一个请求中的校区代号,然后选择楼栋继续抓包,
http://h5cloud.17wanxiao.com:8080/CloudPayment/user/getRoom.do?payProId=1567&schoolcode=786&optype=4&areaid=4&buildid=246&unitid=0&levelid=-1&businesstype=2

这个请求时获取房间号和房间id的,这里结果就不贴了(数据太多了),这个请求中buildid参数就是上个请求中的id, 继续选择房间号然后抓包
http://h5cloud.17wanxiao.com:8080/CloudPayment/user/getRoomState.do?payProId=1888&schoolcode=786&businesstype=2&roomverify=4-246-254--2204

结果
{"returncode":"100","returnmsg":"SUCCESS","quantity":"324.82","quantityunit":"度","canbuy":"true","description":"412"}

【湖南工业大学网站|湖南工业大学电费数据抓取(完美校园版)】这个请求就是真正的电费接口了,这个接口中roomverify就是上个请求中的id,好了整个的电费查询流程就这样,不难发现我们只要有房间号的id和学校的代号就可以获取宿舍的电费了,那么我建议要做电费查询的同学先将学校所有的房间号爬下来存到自己的数据库里,这样我们只要进行最后一个请求就可以获取到电费了,还有就是所有请求中payProId参数最好不要重复,随机一个数就行了,免得爬多了被封

    推荐阅读