1.准备工具
- 抖音app（app下载连接) 提取码：6666
https://github.com/YotaGit/AlgorithmRestore
2.本次主要演示抓取某个话题下的数据 没有图片的话是将图片过滤掉了，这样加载会更快
3.先抓包 话题首页
GET http://aweme.snssdk.com/aweme/v1/challenge/aweme/?ch_id=1603132116358228&query_type=0&cursor=0&count=20&type=5&retry_type=retry_http&iid=361063819794455&device_id=1126379551206542&ac=wifi&channel=yingyonghui&aid=2329&app_name=douyin_lite&version_code=180&version_name=1.8.0&device_platform=android&ssmix=a&device_type=Nexus+6P&device_brand=google&language=zh&os_api=25&os_version=7.1.1&openudid=df2eb2a465f9b18f&manifest_version_code=180&resolution=1440*2392&dpi=560&update_version_code=1800&_rticket=771495793&ts=1630245339&as=aac81339d8000bc5a7c813&cp=3a28c81339d8c8133a2032&mas=0133332319a3f3f3b323f9b9b93fef9cdaf3b323f9d3b993f97933 HTTP/1.1
Accept-Encoding: gzip
X-SS-QUERIES: dGMCA76ot3awALq29Cjed6Wxk9vvcxlI%2BcrVbsSshXPE367hsCodObDobByuYNDsyCWTS76ot3awAOm2rKzdv5e1E7B4tcJ4rILc5cqsABMctdUrsE%2BsYPhZcFX06I%2F6XKH1ew%3D%3D
X-SS-REQ-TICKET: 771495805
Cookie: 你自己的cookie
X-Gorgon: 03742f2b00051aeec7dc6b6c24d8a111044edde107abede12ba7
X-Khronos: 771495
Host: aweme.snssdk.com
Connection: Keep-Alive
User-Agent: okhttp/3.10.0.1
翻页后的请求与响应
GET http://aweme.snssdk.com/aweme/v1/challenge/aweme/?ch_id=1603132116358228&query_type=0&cursor=20&count=20&type=5&retry_type=retry_http&iid=361063819794455&device_id=1126379551206542&ac=wifi&channel=yingyonghui&aid=2329&app_name=douyin_lite&version_code=180&version_name=1.8.0&device_platform=android&ssmix=a&device_type=Nexus+6P&device_brand=google&language=zh&os_api=25&os_version=7.1.1&openudid=df2eb2a465f9b18f&manifest_version_code=180&resolution=1440*2392&dpi=560&update_version_code=1800&_rticket=771550466&ts=1630245393&as=aac81339d8000bc5dec813&cp=3a28c81339d8c8133a2032&mas=0133332319a3f3f3b323f9b9b93fef9c44f3b323f9991993f97933 HTTP/1.1
Accept-Encoding: gzip
X-SS-QUERIES: dGMCA76ot3awALq29Cjed6Wxk9vvcxlI%2BcrVbsSshXPE367hsCodObDobByuYNDsyCWTS76ot3awAOm2rKzdv5e1E7B4tcJ4rILc5cqsABMctdUrsE%2BsYPhZcFX06I%2F6XKH1ew%3D%3D
X-SS-REQ-TICKET: 771550474
Cookie:你自己的cookie
X-Gorgon: 03742f2b000547608bad6b6c24d8a111044edde107abede1b5f4
X-Khronos: 771550
Host: aweme.snssdk.com
Connection: Keep-Alive
User-Agent: okhttp/3.10.0.1
跑一段时间你会发现出不了数据
逐步替换url请求参数会发现是as cp mas这三个参数失效了，得逆向分析一波
4.逆向分析步骤 查壳
查壳的方式很多种，可以先用apktool解开查看特侦，如果应用被加固了，那么需要脱壳
脱壳的教程很多，可以优先考虑FDex2
文章图片
5.jadx使用搜索大法搜索as cp mas三个参数 搜索需要讲究技巧，先搜索as和cp会直接死亡
文章图片
在搜索时按照java的尿性会有以下几种形式(仅供参考)先找长字符的搜索
"mas
"mas=
"mas"
="mas"
果然给蒙对了
文章图片
双击进入发现就是我们想要的
文章图片
上图的代码如下
package com.ss.android.ugc.aweme.app.a;
import android.text.TextUtils;
import com.meituan.robust.ChangeQuickRedirect;
import com.meituan.robust.PatchProxy;
import com.meituan.robust.PatchProxyResult;
import com.meizu.cloud.pushsdk.notification.model.AdvanceSetting;
import com.ss.android.common.applog.EagleEye;
import com.ss.android.common.applog.GlobalContext;
import com.ss.android.common.applog.UserInfo;
import com.ss.android.deviceregister.a;
import com.ss.android.ugc.aweme.app.AwemeApplication;
import com.ss.android.ugc.aweme.app.b.e;
import com.ss.sys.ces.out.ISdk;
import com.ss.sys.ces.out.StcSDKFactory;
import e.t;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.util.ArrayList;
import java.util.List;
/* compiled from: ApiAntiSpamUtils */
public final class b {/* renamed from: areason: collision with root package name */
public static ChangeQuickRedirect f13081a;
public static t a(t tVar, List list, int i) {
String str;
PatchProxyResult proxy = PatchProxy.proxy(new Object[]{tVar, list, new Integer(i)}, null, f13081a, true, 866, new Class[]{t.class, List.class, Integer.TYPE}, t.class);
if (proxy.isSupported) {
return (t) proxy.result;
}
String decode = URLDecoder.decode(tVar.toString());
String b2 = a.b();
if (b2 == null) {
b2 = "";
}
if (decode.contains("&device_id=") || decode.contains("?device_id=")) {
str = UserInfo.getUserInfo(i, decode, (String[]) list.toArray(new String[list.size()]), b2);
} else {
str = UserInfo.getUserInfo(i, decode, (String[]) list.toArray(new String[list.size()]), "");
}
t.a k = tVar.k();
int length = str.length();
if (TextUtils.isEmpty(str)) {
k.a(AdvanceSetting.ADVANCE_SETTING, "a1iosdfgh").a("cp", "androide1");
} else if (length % 2 == 0) {
int i2 = length >> 1;
String substring = str.substring(0, i2);
ISdk sdk = StcSDKFactory.getSDK(GlobalContext.getContext(), (long) AwemeApplication.p().m());
sdk.setSession(e.a());
k.a(AdvanceSetting.ADVANCE_SETTING, substring).a("cp", str.substring(i2, length)).a("mas", EagleEye.byteArrayToHexStr(sdk.encode(substring.getBytes())));
} else {
k.a(AdvanceSetting.ADVANCE_SETTING, "a1qwert123").a("cp", "cbfhckdckkde1");
}
return k.b();
}public static List a(String str) {
int i = 0;
PatchProxyResult proxy = PatchProxy.proxy(new Object[]{str}, null, f13081a, true, 867, new Class[]{String.class}, List.class);
if (proxy.isSupported) {
return (List) proxy.result;
}
ArrayList arrayList = new ArrayList();
while (i <= str.length()) {
int indexOf = str.indexOf(38, i);
if (indexOf == -1) {
indexOf = str.length();
}
int indexOf2 = str.indexOf(61, i);
if (indexOf2 == -1 || indexOf2 > indexOf) {
arrayList.add(URLDecoder.decode(str.substring(i, indexOf)));
arrayList.add(null);
} else {
arrayList.add(URLDecoder.decode(str.substring(i, indexOf2)));
arrayList.add(URLDecoder.decode(str.substring(indexOf2 + 1, indexOf)));
}
i = indexOf + 1;
}
return arrayList;
}public static String b(String str) {
PatchProxyResult proxy = PatchProxy.proxy(new Object[]{str}, null, f13081a, true, 868, new Class[]{String.class}, String.class);
if (proxy.isSupported) {
return (String) proxy.result;
}
if (TextUtils.isEmpty(str)) {
return str;
}
try {
return URLDecoder.decode(str, "UTF-8");
} catch (UnsupportedEncodingException e2) {
com.google.b.a.a.a.a.a.a(e2);
return URLDecoder.decode(str);
}
}
}
这里有一个AdvanceSetting.ADVANCE_SETTING
我们查找一下声明
文章图片
这一节我们只做fridaRpc调用下一节我们直接用python还原这三个参数
文章图片
现在我们已经很清楚需要hook UserInfo.getUserInfo()这个方法
import os
import sysimport fridadef on_message(message, data):
if message['type'] == 'send':
print("[*] {0}".format(message['payload']))
else:
print(message)js_code = """
Java.perform(function(){
var UserInfo = Java.use('com.ss.android.common.applog.UserInfo');
UserInfo.getUserInfo.overload('int', 'java.lang.String', '[Ljava.lang.String;
', 'java.lang.String').implementation=function(a,b,c,d){
send('a:'+a);
send('b:'+b);
send('c:'+c);
send('d:'+d);
user=this.UserInfo(a,b,c,d);
return user;
}
});
"""process = frida.get_remote_device().attach('com.ss.android.ugc.aweme.lite')
script = process.create_script(js_code)
script.on('message', on_message)
script.load()
sys.stdin.read()
getUserInfo的各个参数如下
文章图片
第一个是时间戳
第二个是你传入的请求url
第三个是请求的参数构成的字符串
第四个是一堆数字其实就是deviceId,从下面这个if else可以得知
文章图片
frida.rpc hook模板如下
hook_code = '''
rpc.exports = {
getSign: function(str){
send('getSign');
Java.perform(function(){}
)
}
};
'''
于是结合djax代码可的代码如下
def on_message(message, data):
if message['type'] == 'send':
print("[*] {0}".format(message['payload']))
else:
print(message)js_code = """
rpc.exports = {
getsign: function (timestamps, base_url, params, device_id) {
var sig = "aa";
Java.perform(
function () {
send('getsign');
var StcSDKFactory = Java.use('com.ss.sys.ces.out.StcSDKFactory');
var GlobalContext = Java.use('com.ss.android.common.applog.GlobalContext');
var AwemeApplication = Java.use('com.ss.android.ugc.aweme.app.AwemeApplication');
var EagleEye = Java.use('com.ss.android.common.applog.EagleEye');
var sdk = StcSDKFactory.getSDK(GlobalContext.getContext(), AwemeApplication.p().m());
var UserInfo = Java.use('com.ss.android.common.applog.UserInfo');
var aac=params.toArray(params.length);
var user_info = UserInfo.getUserInfo(timestamps,base_url,params,device_id);
var as = user_info.substring(0, 22);
var cp = user_info.substring(22, user_info.length);
var mas = EagleEye.byteArrayToHexStr(sdk.encode(getBytes(as)));
sig = '&as='+as+'&cp='+cp+'&mas='+mas;
}
)
return sig;
}
}function getBytes(s) {
var bytes = [];
for (var i = 0;
i < s.length;
i++) {
bytes.push(s.charCodeAt(i));
}
return bytes;
}"""process = frida.get_remote_device().attach('com.ss.android.ugc.aweme.lite')
script = process.create_script(js_code)
script.on('message', on_message)
script.load()a= int(time.time())b="https://aweme-eagle.snssdk.com/aweme/v1/feed/?type=0&max_cursor=0&min_cursor=0&count=6&volume=0.0&pull_type=1&need_relieve_aweme=0&filter_warn=0&ts=1630343714&app_type=lite&os_api=25&device_platform=android&device_type=Xiaomi&iid=361063819794455&ssmix=a&manifest_version_code=180&dpi=560&version_code=180&app_name=douyin_lite&version_name=1.8.0&openudid=df2eb2a465f9b18f&device_id=1126379551206542&resolution=1440*2392&os_version=7.1.1&language=zh&device_brand=google&ac=wifi&update_version_code=1800&aid=2329&channel=yingyonghui&_rticket=1630343715767"c="type,0,max_cursor,0,min_cursor,0,count,6,volume,0.0,pull_type,1,need_relieve_aweme,0,filter_warn,0,ts,1630343714,app_type,lite,os_api,25,device_platform,android,device_type,Xiaomi,iid,361063819794455,ssmix,a,manifest_version_code,180,dpi,560,version_code,180,app_name,douyin_lite,version_name,1.8.0,openudid,df2eb2a465f9b18f,device_id,1126379551206542,resolution,1440*2392,os_version,7.1.1,language,zh,device_brand,google,ac,wifi,update_version_code,1800,aid,2329,channel,yingyonghui,_rticket,1630343715767"d="1126379551206542"
sig = script.exports.getsign(a, b, c, d)
print(sig)
【逆向工程|抖音app协议逆向分析】最终的结果如下图所示，每一次的调用都是生成不同的参数
文章图片
推荐阅读
- python|JS逆向之抖音登陆及验证码绕过
- 爬虫案例合集|谷歌地图案例
- 抖音-python|抖音新版signature分析
- ios|盖茨砸数亿美元阻挠推特被收购（马斯克（卑劣的行为））
- 爬虫|让我们用python来采集数据看看找工作都要会什么吧~
- 爬虫|python采集百度图片数据呐~没有你采集不了得图，只有你不想要
- android|Android studio开发APP通过esp8266WiFi控制51单片机LED灯亮灭
- Android系统自带的android.util.Base64的实现源码
- Android面试题|OkHttp原理解析之连接拦截器