Android Netroid解析之——断点续传下载及问题修正 _android

世事洞明皆学问，人情练达即文章。这篇文章主要讲述Android Netroid解析之——断点续传下载及问题修正相关的知识，希望能为你提供帮助。
提到Netroid也许非常多人不知道这个框架，但我假设说Volley想必没有人不知道吧。
Netroid是一个基于Volley实现的android Http库。提供运行网络请求、缓存返回结果、批量图片载入、大文件断点下载的常见Http交互功能，关于网络请求，图片载入没什么好说的，Volley已经有非常多人解析过了，这里来说一下大文件断点下载。
关于大文件断点下载，网上也有非常多实现的demo,为什么要单单说Netroid呢？由于Netroid断点续传不依赖数据库，我在网上看到过非常多的断点续传的样例，无一例外都是依赖于数据库。包含DownloadManager，大名鼎鼎的xutils,可是这两个都有一定的问题。

1.DownloadManager在三星手机上必须打开下载管理才干应用，而打开这个管理必须须要手动打开，普通情况下无伤大雅。视情况而定
2.xutils这个框架别的不知道。文件下载这块慎用

好了。进入正题，Netroid的地址:https://github.com/vince-styling/,以下简单的说一下这个框架文件下载的实现和原理,

// 1 RequestQueue queue = Netroid.newRequestQueue(getApplicationContext(), null); // 2 mDownloder = new FileDownloader(queue, 1) { @Override public FileDownloadRequest buildRequest(String storeFilePath, String url) { return new FileDownloadRequest(storeFilePath, url) { @Override public void prepare() { addHeader(" Accept-Encoding" , " identity" ); super.prepare(); } }; } }; // 3 task.controller = mDownloder.add(mSaveDirPath + task.storeFileName, task.url, new Listener< Void> () { @Override public void onPreExecute() { task.invalidate(); }@Override public void onSuccess(Void response) { showToast(task.storeFileName + " Success!" ); }@Override public void onError(NetroidError error) { NetroidLog.e(error.getMessage()); }@Override public void onFinish() { NetroidLog.e(" onFinish size : " + Formatter.formatFileSize( FileDownloadActivity.this, new File(mSaveDirPath + task.storeFileName).length())); task.invalidate(); }@Override public void onProgressChange(long fileSize, long downloadedSize) { task.onProgressChange(fileSize, downloadedSize); //NetroidLog.e(" ---- fileSize : " + fileSize + " downloadedSize : " + downloadedSize); } });

实现的话非常easy，主要分为三步就能够了
1.创建一个请求队列
2.构建一个文件下载管理器
3.将下载任务加入到队列
如今依据上面的三步来看一下它的实现原理：
第一步：创建一个请求队列:RequestQueue queue = Netroid.newRequestQueue(getApplicationContext(), null);

/** * Creates a default instance of the worker pool and calls {@link RequestQueue#start()} on it. * @param context A {@link Context} to use for creating the cache dir. * @return A started {@link RequestQueue} instance. */ public static RequestQueue newRequestQueue(Context context, DiskCache cache) { int poolSize = RequestQueue.DEFAULT_NETWORK_THREAD_POOL_SIZE; HttpStack stack; String userAgent = " netroid/0" ; try { String packageName = context.getPackageName(); PackageInfo info = context.getPackageManager().getPackageInfo(packageName, 0); userAgent = packageName + " /" + info.versionCode; } catch (NameNotFoundException e) { }if (Build.VERSION.SDK_INT > = Build.VERSION_CODES.GINGERBREAD) { stack = new HurlStack(userAgent, null); } else { // Prior to Gingerbread, HttpUrlConnection was unreliable. // See: http://android-developers.blogspot.com/2011/09/androids-http-clients.html stack = new HttpClientStack(userAgent); } //实例化BasicNetwork，主要用于运行下载请求 Network network = new BasicNetwork(stack, HTTP.UTF_8); //创建请求队列 RequestQueue queue = new RequestQueue(network, poolSize, cache); //非常重要的一步 queue.start(); return queue; }

com.duowan.mobile.netroid.RequestQueue.start():

/** * Starts the dispatchers in this queue. */ public void start() { stop(); // Make sure any currently running dispatchers are stopped. // Create the cache dispatcher and start it. mCacheDispatcher = new CacheDispatcher(mCacheQueue, mNetworkQueue, mCache, mDelivery); mCacheDispatcher.start(); // Create network dispatchers (and corresponding threads) up to the pool size. for (int i = 0; i < mDispatchers.length; i++) { //一个线程，从请求队列中获取任务并运行 NetworkDispatcher networkDispatcher = new NetworkDispatcher(mNetworkQueue, mNetwork, mCache, mDelivery); mDispatchers[i] = networkDispatcher; //Thread run() networkDispatcher.start(); } }/** * Stops the cache and network dispatchers. */ public void stop() { if (mCacheDispatcher != null) { mCacheDispatcher.quit(); } for (NetworkDispatcher mDispatcher : mDispatchers) { //Thread interrupt()线程中断 if (mDispatcher != null) mDispatcher.quit(); } }

框架中对于文件是没有缓存机制的。所以mCacheDispatcher能够不用理它。看一下NetworkDispatcher这个线程做了什么：com.duowan.mobile.netroid.NetworkDispatcher

public class NetworkDispatcher extends Thread {@Override public void run() { //设置线程优先级 Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND); Request request; while (true) { try { // Take a request from the queue.假设队列为空，则堵塞 request = mQueue.take(); } catch (InterruptedException e) { // We may have been interrupted because it was time to quit.唯有线程中断的时候mQuit才为true,InterruptedException为中断异常 //mQueue.take()假设队列为null,仅仅会堵塞，不会跑出异常 if (mQuit) return; continue; }try { request.addMarker(" network-queue-take" ); //准备运行 mDelivery.postPreExecute(request); // If the request was cancelled already, // do not perform the network request. if (request.isCanceled()) { request.finish(" network-discard-cancelled" ); mDelivery.postCancel(request); mDelivery.postFinish(request); continue; }// Perform the network request.最重要一步。Netroid实例化的BasicNetwork在这里运行网络请求 NetworkResponse networkResponse = mNetwork.performRequest(request); request.addMarker(" network-http-complete" ); // Parse the response here on the worker thread.重命名一下。没做什么 Response< ?> response = request.parseNetworkResponse(networkResponse); request.addMarker(" network-parse-complete" ); // Write to cache if applicable. if (mCache != null & & request.shouldCache() & & response.cacheEntry != null) { response.cacheEntry.expireTime = request.getCacheExpireTime(); mCache.putEntry(request.getCacheKey(), response.cacheEntry); request.addMarker(" network-cache-written" ); }// Post the response back. request.markDelivered(); mDelivery.postResponse(request, response); } catch (NetroidError netroidError) { mDelivery.postError(request, request.parseNetworkError(netroidError)); } catch (Exception e) { NetroidLog.e(e, " Unhandled exception %s" , e.toString()); mDelivery.postError(request, new NetroidError(e)); } } }}

这里最重要的一步就是NetworkResponse networkResponse = mNetwork.performRequest(request); 运行网络请求，可是我们不要忘记我们的mQueue还是空的，mQueue.take()正在堵塞着呢，所以，如今还没有办法进行网络请求，因此我们须要在mQueue中填充任务，才干进行我们的网络请求。
不要忘记这里哦。由于我们还会回到这里！

第二步：创建一个文件下载管理器：new FileDownloader(queue, 1)

mDownloder = new FileDownloader(queue, 1) { @Override public FileDownloadRequest buildRequest(String storeFilePath, String url) { return new FileDownloadRequest(storeFilePath, url) { @Override public void prepare() { addHeader(" Accept-Encoding" , " identity" ); super.prepare(); } }; } };

这里有没有看着非常吓人，我起初看的时候也吓了一跳，事实上就是实例化的时候，顺手override了一下
【Android Netroid解析之——断点续传下载及问题修正】

/** The parallel task count, recommend less than 3. */ private final int mParallelTaskCount; /** The linked Task Queue. */ private final LinkedList< DownloadController> mTaskQueue; /** * Construct Downloader and init the Task Queue. * @param queue The RequestQueue for dispatching Download task. * @param parallelTaskCount *Allows parallel task count, *don' t forget the value must less than ThreadPoolSize of the RequestQueue. */ public FileDownloader(RequestQueue queue, int parallelTaskCount) { if (parallelTaskCount > = queue.getThreadPoolSize()) { throw new IllegalArgumentException(" parallelTaskCount[" + parallelTaskCount + " ] must less than threadPoolSize[" + queue.getThreadPoolSize() + " ] of the RequestQueue." ); }mTaskQueue = new LinkedList< DownloadController> (); mParallelTaskCount = parallelTaskCount; mRequestQueue = queue; }

这里是须要注意的一点，mParallelTaskCount并发的数量最好< 3.
第三步：将下载任务加入到队列，task.controller = mDownloder.add(mSaveDirPath + task.storeFileName, task.url, new Listener< Void> ()：

/** * Create a new download request, this request might not run immediately because the parallel task limitation, * you can check the status by the {@link DownloadController} which you got after invoke this method. * * Note: don' t perform this method twice or more with same parameters, because we didn' t check for * duplicate tasks, it rely on developer done. * * Note: this method should invoke in the main thread. * * @param storeFilePath Once download successed, we' ll find it by the store file path. * @param url The download url. * @param listener The event callback by status; * @return The task controller allows pause or resume or discard operation. */ public DownloadController add(String storeFilePath, String url, Listener< Void> listener) { // only fulfill requests that were initiated from the main thread.(reason for the Delivery?) //看名字就知道 throwIfNotOnMainThread(); //创建一个下载控制器 DownloadController controller = new DownloadController(storeFilePath, url, listener); synchronized (mTaskQueue) { //这可不是mQueue,这里仅仅是一个DownloadController的LinkedList集合 mTaskQueue.add(controller); } //重点来了 schedule(); return controller; }

/** * Traverse the Task Queue, count the running task then deploy more if it can be. */ private void schedule() { // make sure only one thread can manipulate the Task Queue. synchronized (mTaskQueue) { // counting ran task. int parallelTaskCount = 0; for (DownloadController controller : mTaskQueue) { //累计队列中正在下载的的任务数 if (controller.isDownloading()) parallelTaskCount++; } //当正在下载的个数大于并行任务数的时候，不在运行下载任务 /* * 这里举个样例说明一下：我们默认mParallelTaskCount=1 * 当我们加入第一个任务的时候。这个的controller.isDownloading()肯定是false * 所以parallelTaskCount > = mParallelTaskCount是不成立的，当我们再加入一个任务的时候，如今mTaskQueue.size是2了 * 且第一个isDownloading，为了保证并发数量为1，会return,说的有点乱。不知道说明确了没有 */ if (parallelTaskCount > = mParallelTaskCount) return; // try to deploy all Task if they' re await. for (DownloadController controller : mTaskQueue) { //deploy()，将任务加入到队列中 if (controller.deploy() & & ++parallelTaskCount == mParallelTaskCount) return; } } }

/** * For the parallel reason, only the {@link FileDownloader#schedule()} can call this method. * @return true if deploy is successed. */ private boolean deploy() { if (mStatus != STATUS_WAITING) return false; //第二步我说非常吓人那个地方 mRequest = buildRequest(mStoreFilePath, mUrl); // we create a Listener to wrapping that Listener which developer specified, // for the onFinish(), onSuccess(), onError() won' t call when request was cancel reason. mRequest.setListener(new Listener< Void> () { boolean isCanceled; @Override public void onPreExecute() { mListener.onPreExecute(); }@Override public void onFinish() { // we don' t inform FINISH when it was cancel. if (!isCanceled) { mStatus = STATUS_PAUSE; mListener.onFinish(); // when request was FINISH, remove the task and re-schedule Task Queue. //remove(DownloadController.this); } }@Override public void onSuccess(Void response) { // we don' t inform SUCCESS when it was cancel. if (!isCanceled) { mListener.onSuccess(response); mStatus = STATUS_SUCCESS; remove(DownloadController.this); } }@Override public void onError(NetroidError error) { // we don' t inform ERROR when it was cancel. if (!isCanceled) mListener.onError(error); }@Override public void onCancel() { mListener.onCancel(); isCanceled = true; }@Override public void onProgressChange(long fileSize, long downloadedSize) { mListener.onProgressChange(fileSize, downloadedSize); } }); mStatus = STATUS_DOWNLOADING; //我擦，最终把任务加到队列中了 mRequestQueue.add(mRequest); return true; }

mRequestQueue.add(mRequest); 任务加到队列中了，都到了这里了看一下怎么加的吧

public Request add(Request request) { // Tag the request as belonging to this queue and add it to the set of current requests. request.setRequestQueue(this); synchronized (mCurrentRequests) { mCurrentRequests.add(request); }// Process requests in the order they are added. request.setSequence(getSequenceNumber()); request.addMarker(" add-to-queue" ); // If the request is uncacheable or forceUpdate, skip the cache queue and go straight to the network. if (request.isForceUpdate() || !request.shouldCache()) { mDelivery.postNetworking(request); mNetworkQueue.add(request); return request; }}

request.shouldCache()有兴趣的能够自己去看一下。这里说明了文件下载没有缓存机制，这里就不多说了，由于假设你还没有忘记的话。mQueue.take()还在堵塞着呢。好了让我们回到第一步，运行网络请求
NetworkResponse networkResponse = mNetwork.performRequest(request);

@Override public NetworkResponse performRequest(Request< ?> request) throws NetroidError { // Determine if request had non-http perform. NetworkResponse networkResponse = request.perform(); if (networkResponse != null) return networkResponse; long requestStart = SystemClock.elapsedRealtime(); while (true) { // If the request was cancelled already, // do not perform the network request. if (request.isCanceled()) { request.finish(" perform-discard-cancelled" ); mDelivery.postCancel(request); throw new NetworkError(networkResponse); }HttpResponse httpResponse = null; byte[] responseContents = null; try { // prepare to perform this request, normally is reset the request headers. request.prepare(); httpResponse = mHttpStack.performRequest(request); StatusLine statusLine = httpResponse.getStatusLine(); int statusCode = statusLine.getStatusCode(); responseContents = request.handleResponse(httpResponse, mDelivery); if (statusCode < 200 || statusCode > 299) throw new IOException(); // if the request is slow, log it. long requestLifetime = SystemClock.elapsedRealtime() - requestStart; logSlowRequests(requestLifetime, request, responseContents, statusLine); return new NetworkResponse(statusCode, responseContents, parseCharset(httpResponse)); } catch (SocketTimeoutException e) { attemptRetryOnException(" socket" , request, new TimeoutError()); } catch (ConnectTimeoutException e) { attemptRetryOnException(" connection" , request, new TimeoutError()); } catch (MalformedURLException e) { throw new RuntimeException(" Bad URL " + request.getUrl(), e); } catch (IOException e) { if (httpResponse == null) throw new NoConnectionError(e); int statusCode = httpResponse.getStatusLine().getStatusCode(); NetroidLog.e(" Unexpected response code %d for %s" , statusCode, request.getUrl()); if (responseContents != null) { networkResponse = new NetworkResponse(statusCode, responseContents, parseCharset(httpResponse)); if (statusCode == HttpStatus.SC_UNAUTHORIZED || statusCode == HttpStatus.SC_FORBIDDEN) { attemptRetryOnException(" auth" , request, new AuthFailureError(networkResponse)); } else { // TODO: Only throw ServerError for 5xx status codes. throw new ServerError(networkResponse); } } else { throw new NetworkError(networkResponse); } } } }

这里我给改了一下，详细的能够看一下作者的。他有一块dead code,网络请求这一块没什么好说的。可是这里有一句非常重要的代码
responseContents = request.handleResponse(httpResponse, mDelivery); 。写文件,断点续传的原理

/** * In this method, we got the Content-Length, with the TemporaryFile length, * we can calculate the actually size of the whole file, if TemporaryFile not exists, * we' ll take the store file length then compare to actually size, and if equals, * we consider this download was already done. * We used {@link RandomAccessFile} to continue download, when download success, * the TemporaryFile will be rename to StoreFile. */ @Override public byte[] handleResponse(HttpResponse response, Delivery delivery) throws IOException, ServerError { // Content-Length might be negative when use HttpURLConnection because it default header Accept-Encoding is gzip, // we can force set the Accept-Encoding as identity in prepare() method to slove this problem but also disable gzip response. HttpEntity entity = response.getEntity(); //获取文件的总大小 long fileSize = entity.getContentLength(); if (fileSize < = 0) { NetroidLog.d(" Response doesn' t present Content-Length!" ); }long downloadedSize = mTemporaryFile.length(); /* * 是否支持断点续传 * * client每次提交下载请求时。服务端都要加入这两个响应头，以保证client和服务端将此下载识别为能够断点续传的下载： *Accept-Ranges：告知下载client这是一个能够恢复续传的下载，存放本次下载的開始字节位置、文件的字节大小； *ETag：保存文件的唯一标识（我在用的文件名称+文件最后改动时间，以便续传请求时对文件进行验证）。 *Last-Modified：可选响应头，存放服务端文件的最后改动时间，用于验证 */ boolean isSupportRange = HttpUtils.isSupportRange(response); if (isSupportRange) { fileSize += downloadedSize; // Verify the Content-Range Header, to ensure temporary file is part of the whole file. // Sometime, temporary file length add response content-length might greater than actual file length, // in this situation, we consider the temporary file is invalid, then throw an exception. String realRangeValue = https://www.songbingjia.com/android/HttpUtils.getHeader(response, " Content-Range" ); // response Content-Range may be null when " Range=bytes=0-" if (!TextUtils.isEmpty(realRangeValue)) { String assumeRangeValue = " bytes " + downloadedSize + " -" + (fileSize - 1); if (TextUtils.indexOf(realRangeValue, assumeRangeValue) == -1) { throw new IllegalStateException( " The Content-Range Header is invalid Assume[" + assumeRangeValue + " ] vs Real[" + realRangeValue + " ], " + " please remove the temporary file [" + mTemporaryFile + " ]." ); } } }// Compare the store file size(after download successes have) to server-side Content-Length. // temporary file will rename to store file after download success, so we compare the // Content-Length to ensure this request already download or not. if (fileSize > 0 & & mStoreFile.length() == fileSize) { // Rename the store file to temporary file, mock the download success. ^_^ mStoreFile.renameTo(mTemporaryFile); // Deliver download progress. delivery.postDownloadProgress(this, fileSize, fileSize); return null; } //之所以能够实现断点续传的原因所在 RandomAccessFile tmpFileRaf = new RandomAccessFile(mTemporaryFile, " rw" ); // If server-side support range download, we seek to last point of the temporary file. if (isSupportRange) { //移动文件读写指针位置 tmpFileRaf.seek(downloadedSize); } else { // If not, truncate the temporary file then start download from beginning. tmpFileRaf.setLength(0); downloadedSize = 0; }try { InputStream in = entity.getContent(); // Determine the response gzip encoding, support for HttpClientStack download. if (HttpUtils.isGzipContent(response) & & !(in instanceof GZIPInputStream)) { in = new GZIPInputStream(in); } byte[] buffer = new byte[6 * 1024]; // 6K buffer int offset; while ((offset = in.read(buffer)) != -1) { //写文件 tmpFileRaf.write(buffer, 0, offset); downloadedSize += offset; long currTime = SystemClock.uptimeMillis(); //控制下载进度的速度 if (currTime - lastUpdateTime > = DEFAULT_TIME) { lastUpdateTime = currTime; delivery.postDownloadProgress(this, fileSize, downloadedSize); }if (isCanceled()) { delivery.postCancel(this); break; } } } finally { try { // Close the InputStream and release the resources by " consuming the content" . if (entity != null) entity.consumeContent(); } catch (Exception e) { // This can happen if there was an exception above that left the entity in // an invalid state. NetroidLog.v(" Error occured when calling consumingContent" ); } tmpFileRaf.close(); }return null; }

实现断点续传主要靠的RandomAccessFile,你假设对c语言不陌生的话tmpFileRaf.seek(downloadedSize)和int fseek(FILE *stream, long offset, int fromwhere); 是不是有点眼熟，仅仅与RandomAccessFile就不说了。

好了，Netroid的原理基本上就是这些了，讲一下我用的时候遇到的两个问题：
1.下载进度的速度太快。你假设用notifition来显示，会出现ANR,所以我们要控制一下它的速度，详细方法在上面

//控制下载进度的速度 if (currTime - lastUpdateTime > = DEFAULT_TIME) { lastUpdateTime = currTime; delivery.postDownloadProgress(this, fileSize, downloadedSize); }

2.第二个问题是当你下载的时候。假设把WiFi关掉。即使没下完。也会被标记为done,改动主要是在在FileDownloader.DownloadController的deploy()中

@Override public void onFinish() { // we don' t inform FINISH when it was cancel. if (!isCanceled) { mStatus = STATUS_PAUSE; mListener.onFinish(); // when request was FINISH, remove the task and re-schedule Task Queue. //remove(DownloadController.this); } }@Override public void onSuccess(Void response) { // we don' t inform SUCCESS when it was cancel. if (!isCanceled) { mListener.onSuccess(response); mStatus = STATUS_SUCCESS; remove(DownloadController.this); } }

把onFinish的status改成STATUS_PAUSE。并去掉remove(DownloadController.this); 。在onSuccess中再将status改动为STATUS_SUCCESS，并remove，当然这个办法治标不治本。假设有谁知道请告之，谢谢。