Android|Android 12(S) 图形显示系统 - BufferQueue的工作流程(十一)

题外话
我竟然已经写了这个系列的十一篇文章了,虽然内容很浅显,虽然内容很枯燥,虽然内容也许没营养,但我为自己的坚持点赞!
Android|Android 12(S) 图形显示系统 - BufferQueue的工作流程(十一)
文章图片

一、前言 前面的两篇文章,分别讲解了Producer的处理逻辑和queue buffer后通过FrameAvailableListener通知到Consumer的基本过程。
流程已经走到了BufferQueueConsumer::acquireBuffer中,所以这篇文章聚焦Consumer的一些处理逻辑。
还是把流程图贴上来
Android|Android 12(S) 图形显示系统 - BufferQueue的工作流程(十一)
文章图片


从流程图中看,这篇文章就是讲解右半部分的内容。

二、消费者-Consumer的相关逻辑 了解了 BufferQueueCore 和 BufferQueueProducer,接着看 BufferQueue 的最后一个元素:BufferQueueConsumer。
BufferQueueConsumer作为消费者的一个代表元素通过 acquireBuffer 来获取图像缓冲区,通过 releaseBuffer 来释放该缓冲区。
下面就分别看看 BufferQueueConsumer 中 acquireBuffer 和 releaseBuffer 两个操作的具体流程。


2.1 代码位置
/frameworks/native/libs/gui/BufferQueueConsumer.cpp

2.2 acquireBuffer的逻辑
先看 acquireBuffer 的过程,上源码

status_t BufferQueueConsumer::acquireBuffer(BufferItem* outBuffer, nsecs_t expectedPresent, uint64_t maxFrameNumber) { ATRACE_CALL(); int numDroppedBuffers = 0; sp listener; { std::unique_lock lock(mCore->mMutex); // Check that the consumer doesn't currently have the maximum number of // buffers acquired. We allow the max buffer count to be exceeded by one // buffer so that the consumer can successfully set up the newly acquired // buffer before releasing the old one.// 检查acquire的buffer的数量是否超出了限制 int numAcquiredBuffers = 0; for (int s : mCore->mActiveBuffers) { if (mSlots[s].mBufferState.isAcquired()) { ++numAcquiredBuffers; } } const bool acquireNonDroppableBuffer = mCore->mAllowExtraAcquire && numAcquiredBuffers == mCore->mMaxAcquiredBufferCount + 1; if (numAcquiredBuffers >= mCore->mMaxAcquiredBufferCount + 1 && !acquireNonDroppableBuffer) { BQ_LOGE("acquireBuffer: max acquired buffer count reached: %d (max %d)", numAcquiredBuffers, mCore->mMaxAcquiredBufferCount); return INVALID_OPERATION; }bool sharedBufferAvailable = mCore->mSharedBufferMode && mCore->mAutoRefresh && mCore->mSharedBufferSlot != BufferQueueCore::INVALID_BUFFER_SLOT; // In asynchronous mode the list is guaranteed to be one buffer deep, // while in synchronous mode we use the oldest buffer. // 检查BufferQueueCore中的mQueue队列是否为空 if (mCore->mQueue.empty() && !sharedBufferAvailable) { return NO_BUFFER_AVAILABLE; } // 获取BufferQueueCore中的mQueue队列的迭代器 BufferQueueCore::Fifo::iterator front(mCore->mQueue.begin()); // If expectedPresent is specified, we may not want to return a buffer yet. // If it's specified and there's more than one buffer queued, we may want // to drop a buffer. // Skip this if we're in shared buffer mode and the queue is empty, // since in that case we'll just return the shared buffer. if (expectedPresent != 0 && !mCore->mQueue.empty()) {// expectedPresent表示期望这个buffer什么时候显示到屏幕上。 // 如果buffer的期望显示时间小于expectedPresent,我们会acquire and return这个buffer // 如果我们不想显示它直到expectedPresent之后,可以返回PRESENT_LATER// The 'expectedPresent' argument indicates when the buffer is expected // to be presented on-screen. If the buffer's desired present time is // earlier (less) than expectedPresent -- meaning it will be displayed // on time or possibly late if we show it as soon as possible -- we // acquire and return it. If we don't want to display it until after the // expectedPresent time, we return PRESENT_LATER without acquiring it. //// 安全起见,如果expectedPresent超过了buffer的期望显示时间1秒,我们会推迟acquire // To be safe, we don't defer acquisition if expectedPresent is more // than one second in the future beyond the desired present time // (i.e., we'd be holding the buffer for a long time). // // NOTE: Code assumes monotonic time values from the system clock // are positive.// 检查是否需要丢弃一些帧,主要是判断timestamps & expectedPresent // Start by checking to see if we can drop frames. We skip this check if // the timestamps are being auto-generated by Surface. If the app isn't // generating timestamps explicitly, it probably doesn't want frames to // be discarded based on them. while (mCore->mQueue.size() > 1 && !mCore->mQueue[0].mIsAutoTimestamp) { const BufferItem& bufferItem(mCore->mQueue[1]); // If dropping entry[0] would leave us with a buffer that the // consumer is not yet ready for, don't drop it. if (maxFrameNumber && bufferItem.mFrameNumber > maxFrameNumber) { break; }// If entry[1] is timely, drop entry[0] (and repeat). We apply an // additional criterion here: we only drop the earlier buffer if our // desiredPresent falls within +/- 1 second of the expected present. // Otherwise, bogus desiredPresent times (e.g., 0 or a small // relative timestamp), which normally mean "ignore the timestamp // and acquire immediately", would cause us to drop frames. // // We may want to add an additional criterion: don't drop the // earlier buffer if entry[1]'s fence hasn't signaled yet. nsecs_t desiredPresent = bufferItem.mTimestamp; // desiredPresent比expectedPresent小了1 second多,或desiredPresent大于expectedPresent if (desiredPresent < expectedPresent - MAX_REASONABLE_NSEC || desiredPresent > expectedPresent) { // This buffer is set to display in the near future, or // desiredPresent is garbage. Either way we don't want to drop // the previous buffer just to get this on the screen sooner. BQ_LOGV("acquireBuffer: nodrop desire=%" PRId64 " expect=%" PRId64 " (%" PRId64 ") now=%" PRId64, desiredPresent, expectedPresent, desiredPresent - expectedPresent, systemTime(CLOCK_MONOTONIC)); break; }BQ_LOGV("acquireBuffer: drop desire=%" PRId64 " expect=%" PRId64 " size=%zu", desiredPresent, expectedPresent, mCore->mQueue.size()); // 处理要drop的buffer if (!front->mIsStale) { // Front buffer is still in mSlots, so mark the slot as free // 对应的BufferSlot设置为FREE状态 mSlots[front->mSlot].mBufferState.freeQueued(); // After leaving shared buffer mode, the shared buffer will // still be around. Mark it as no longer shared if this // operation causes it to be free. if (!mCore->mSharedBufferMode && mSlots[front->mSlot].mBufferState.isFree()) { mSlots[front->mSlot].mBufferState.mShared = false; }// mActiveBuffers :绑定了GraphicBuffer且状态为非FREE的BufferSlot集合; // mFreeBuffers :绑定了GraphicBuffer且状态为FREE的BufferSlot集合;// Don't put the shared buffer on the free list if (!mSlots[front->mSlot].mBufferState.isShared()) { mCore->mActiveBuffers.erase(front->mSlot); // 从mActiveBuffers删除 mCore->mFreeBuffers.push_back(front->mSlot); // 添加进mFreeBuffers }if (mCore->mBufferReleasedCbEnabled) { listener = mCore->mConnectedProducerListener; // 设置生产者的监听器 } ++numDroppedBuffers; // 计数加1,记录drop了几个buffer }mCore->mQueue.erase(front); // 从mQueue中删除 front = mCore->mQueue.begin(); // 重置front,进入下一次while循环 }// See if the front buffer is ready to be acquired nsecs_t desiredPresent = front->mTimestamp; bool bufferIsDue = desiredPresent <= expectedPresent || desiredPresent > expectedPresent + MAX_REASONABLE_NSEC; bool consumerIsReady = maxFrameNumber > 0 ? front->mFrameNumber <= maxFrameNumber : true; if (!bufferIsDue || !consumerIsReady) { BQ_LOGV("acquireBuffer: defer desire=%" PRId64 " expect=%" PRId64 " (%" PRId64 ") now=%" PRId64 " frame=%" PRIu64 " consumer=%" PRIu64, desiredPresent, expectedPresent, desiredPresent - expectedPresent, systemTime(CLOCK_MONOTONIC), front->mFrameNumber, maxFrameNumber); ATRACE_NAME("PRESENT_LATER"); return PRESENT_LATER; }BQ_LOGV("acquireBuffer: accept desire=%" PRId64 " expect=%" PRId64 " " "(%" PRId64 ") now=%" PRId64, desiredPresent, expectedPresent, desiredPresent - expectedPresent, systemTime(CLOCK_MONOTONIC)); } // 走到这里就说明:该丢弃的已经都丢弃了,余下的就可以拿去显示了。 int slot = BufferQueueCore::INVALID_BUFFER_SLOT; if (sharedBufferAvailable && mCore->mQueue.empty()) { // make sure the buffer has finished allocating before acquiring it // 共享Buffer模式下处理 mCore->waitWhileAllocatingLocked(lock); slot = mCore->mSharedBufferSlot; // Recreate the BufferItem for the shared buffer from the data that // was cached when it was last queued. outBuffer->mGraphicBuffer = mSlots[slot].mGraphicBuffer; outBuffer->mFence = Fence::NO_FENCE; outBuffer->mFenceTime = FenceTime::NO_FENCE; outBuffer->mCrop = mCore->mSharedBufferCache.crop; outBuffer->mTransform = mCore->mSharedBufferCache.transform & ~static_cast( NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY); outBuffer->mScalingMode = mCore->mSharedBufferCache.scalingMode; outBuffer->mDataSpace = mCore->mSharedBufferCache.dataspace; outBuffer->mFrameNumber = mCore->mFrameCounter; outBuffer->mSlot = slot; outBuffer->mAcquireCalled = mSlots[slot].mAcquireCalled; outBuffer->mTransformToDisplayInverse = (mCore->mSharedBufferCache.transform & NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY) != 0; outBuffer->mSurfaceDamage = Region::INVALID_REGION; outBuffer->mQueuedBuffer = false; outBuffer->mIsStale = false; outBuffer->mAutoRefresh = mCore->mSharedBufferMode && mCore->mAutoRefresh; } else if (acquireNonDroppableBuffer && front->mIsDroppable) { BQ_LOGV("acquireBuffer: front buffer is not droppable"); return NO_BUFFER_AVAILABLE; } else { // 从front获取对应的slot index slot = front->mSlot; *outBuffer = *front; }ATRACE_BUFFER_INDEX(slot); BQ_LOGV("acquireBuffer: acquiring { slot=%d/%" PRIu64 " buffer=%p }", slot, outBuffer->mFrameNumber, outBuffer->mGraphicBuffer->handle); if (!outBuffer->mIsStale) { mSlots[slot].mAcquireCalled = true; // Don't decrease the queue count if the BufferItem wasn't // previously in the queue. This happens in shared buffer mode when // the queue is empty and the BufferItem is created above. if (mCore->mQueue.empty()) { mSlots[slot].mBufferState.acquireNotInQueue(); } else { // 将BufferState状态改为acquire mSlots[slot].mBufferState.acquire(); } mSlots[slot].mFence = Fence::NO_FENCE; }// If the buffer has previously been acquired by the consumer, set // mGraphicBuffer to NULL to avoid unnecessarily remapping this buffer // on the consumer side if (outBuffer->mAcquireCalled) { outBuffer->mGraphicBuffer = nullptr; } //将该Buffer从mQueue中移除 mCore->mQueue.erase(front); // We might have freed a slot while dropping old buffers, or the producer // may be blocked waiting for the number of buffers in the queue to // decrease. mCore->mDequeueCondition.notify_all(); ATRACE_INT(mCore->mConsumerName.string(), static_cast(mCore->mQueue.size())); #ifndef NO_BINDER mCore->mOccupancyTracker.registerOccupancyChange(mCore->mQueue.size()); #endif VALIDATE_CONSISTENCY(); } // 回调,通知生产者 if (listener != nullptr) { for (int i = 0; i < numDroppedBuffers; ++i) { listener->onBufferReleased(); } }return NO_ERROR; }


acquireBuffer 函数中的逻辑也非常的清晰,源码中也做了详细注释。
主要就是这几件事情:
  1. 判断 BufferQueueCore 中的 mQueue 是否为空,mQueue 就是前面 BufferQueueProducer 调用 queueBuffer 函数时,将缓冲区入队的容器;
  2. 取出对应的 BufferSlot(会有一些判断规则,舍弃一些buffer);
  3. 将 BufferState 改为 acquire 状态;
  4. 将该 Buffer 从 mQueue 中移除;
Android|Android 12(S) 图形显示系统 - BufferQueue的工作流程(十一)
文章图片

2.3 消费者acquire拿到buffer后又是怎样通知release buffer呢?
要回答这个问题,我们需要在回到调用acquireBuffer的地方,即 BLASTBufferQueue::processNextBufferLocked 函数中,先看其代码:
void BLASTBufferQueue::processNextBufferLocked(bool useNextTransaction) { ...... SurfaceComposerClient::Transaction localTransaction; bool applyTransaction = true; SurfaceComposerClient::Transaction* t = &localTransaction; // acquireBuffer获取要处理的buffer BufferItem bufferItem; status_t status = mBufferItemConsumer->acquireBuffer(&bufferItem, 0 /* expectedPresent */, false); ...... // 拿到了实际的GraphicBuffer了 auto buffer = bufferItem.mGraphicBuffer; mNumFrameAvailable--; // 某些情况下,直接releaseBuffer而无需送SurfaceFlinger合成显示mLastAcquiredFrameNumber = bufferItem.mFrameNumber; ReleaseCallbackId releaseCallbackId(buffer->getId(), mLastAcquiredFrameNumber); mSubmitted[releaseCallbackId] = bufferItem; ....// Ensure BLASTBufferQueue stays alive until we receive the transaction complete callback. incStrong((void*)transactionCallbackThunk); // release buffer的回到函数 auto releaseBufferCallback = std::bind(releaseBufferCallbackThunk, wp(this) /* callbackContext */, std::placeholders::_1, std::placeholders::_2, std::placeholders::_3, std::placeholders::_4); t->setBuffer(mSurfaceControl, buffer, releaseCallbackId, releaseBufferCallback); ......if (applyTransaction) { t->setApplyToken(mApplyToken).apply(); } }

上述代码,做了比较多的简化,只保留我认为比较重要的部分。
  1. 调用acquireBuffer获取一个BufferItem;
  2. 取出GraphicBuffer -- auto buffer = bufferItem.mGraphicBuffer;
  3. 通过事务Transaction来向SurfaceFlinger提交Buffer与图层的属性;
t->setBuffer(mSurfaceControl, buffer, releaseCallbackId, releaseBufferCallback);

对于setBuffer,就是设置传递给SF的buffer,并指定了一个releaseBufferCallback,暂时可以理解为SF消费完这个buffer,就会通过这个callback通知来释放这个buffer。
本文作者@二的次方2022-03-23 发布于博客园
在acquireBuffer中加入log,打印调用堆栈信息,如下:
11-13 01:23:59.27530163030 E BufferQueueConsumer: stackdump:#00 pc 000580ff/system/lib/libgui.so (android::BufferQueueConsumer::releaseBuffer(int, unsigned long long, android::sp const&, void*, void*)+130) 11-13 01:23:59.27530163030 E BufferQueueConsumer: stackdump:#01 pc 00059117/system/lib/libgui.so (android::BufferQueueConsumer::releaseBuffer(int, unsigned long long, void*, void*, android::sp const&)+30) 11-13 01:23:59.27530163030 E BufferQueueConsumer: stackdump:#02 pc 00076d27/system/lib/libgui.so (android::ConsumerBase::releaseBufferLocked(int, android::sp, void*, void*)+134) 11-13 01:23:59.27530163030 E BufferQueueConsumer: stackdump:#03 pc 0007580d/system/lib/libgui.so (android::BufferItemConsumer::releaseBuffer(android::BufferItem const&, android::sp const&)+140) 11-13 01:23:59.27530163030 E BufferQueueConsumer: stackdump:#04 pc 0006c467/system/lib/libgui.so (android::BLASTBufferQueue::releaseBufferCallback(android::ReleaseCallbackId const&, android::sp const&, unsigned int, unsigned int)+1362) 11-13 01:23:59.27530163030 E BufferQueueConsumer: stackdump:#05 pc 0006d827/system/lib/libgui.so (android::releaseBufferCallbackThunk(android::wp, android::ReleaseCallbackId const&, android::sp const&, unsigned int, unsigned int)+62) 11-13 01:23:59.27630163030 E BufferQueueConsumer: stackdump:#06 pc 0007039b/system/lib/libgui.so (std::__1::__function::__func const&, std::__1::placeholders::__ph<2> const&, std::__1::placeholders::__ph<3> const&, std::__1::placeholders::__ph<4> const&>, std::__1::allocator const&, std::__1::placeholders::__ph<2> const&, std::__1::placeholders::__ph<3> const&, std::__1::placeholders::__ph<4> const&> >, void (android::ReleaseCallbackId const&, android::sp const&, unsigned int, unsigned int)>::operator()(android::ReleaseCallbackId const&, android::sp const&, 11-13 01:23:59.27630163030 E BufferQueueConsumer: stackdump:#07 pc 000a7d47/system/lib/libgui.so (android::TransactionCompletedListener::onTransactionCompleted(android::ListenerStats)+3382) 11-13 01:23:59.27630163030 E BufferQueueConsumer: stackdump:#08 pc 000925a5/system/lib/libgui.so (int android::SafeBnInterface::callLocalAsync(android::Parcel const&, android::Parcel*, void (android::ITransactionCompletedListener::*)(android::ListenerStats))+204) 11-13 01:23:59.27630163030 E BufferQueueConsumer: stackdump:#09 pc 00028ddb/system/lib/libbinder.so (android::BBinder::transact(unsigned int, android::Parcel const&, android::Parcel*, unsigned int)+162)


看上面的调用栈,是不是一目了然,从遥远的Binder来的神秘信息触发了这一些列的事件:
>>> releaseBufferCallbackThunk
>>> BLASTBufferQueue::releaseBufferCallback
>>> BufferItemConsumer::releaseBuffer
>>> ConsumerBase::releaseBufferLocked
>>> BufferQueueConsumer::releaseBuffer
Android|Android 12(S) 图形显示系统 - BufferQueue的工作流程(十一)
文章图片

2.4 releaseBuffer的逻辑
老规矩,直接看代码,是不是很简单啊!
status_t BufferQueueConsumer::releaseBuffer(int slot, uint64_t frameNumber, const sp& releaseFence, EGLDisplay eglDisplay, EGLSyncKHR eglFence) { ATRACE_CALL(); ATRACE_BUFFER_INDEX(slot); if (slot < 0 || slot >= BufferQueueDefs::NUM_BUFFER_SLOTS || releaseFence == nullptr) { BQ_LOGE("releaseBuffer: slot %d out of range or fence %p NULL", slot, releaseFence.get()); return BAD_VALUE; }sp listener; { // Autolock scope std::lock_guard lock(mCore->mMutex); // If the frame number has changed because the buffer has been reallocated, // we can ignore this releaseBuffer for the old buffer. // Ignore this for the shared buffer where the frame number can easily // get out of sync due to the buffer being queued and acquired at the // same time. if (frameNumber != mSlots[slot].mFrameNumber && !mSlots[slot].mBufferState.isShared()) { return STALE_BUFFER_SLOT; }if (!mSlots[slot].mBufferState.isAcquired()) { BQ_LOGE("releaseBuffer: attempted to release buffer slot %d " "but its state was %s", slot, mSlots[slot].mBufferState.string()); return BAD_VALUE; }mSlots[slot].mEglDisplay = eglDisplay; mSlots[slot].mEglFence = eglFence; mSlots[slot].mFence = releaseFence; mSlots[slot].mBufferState.release(); //置为FREE状态// After leaving shared buffer mode, the shared buffer will // still be around. Mark it as no longer shared if this // operation causes it to be free. if (!mCore->mSharedBufferMode && mSlots[slot].mBufferState.isFree()) { mSlots[slot].mBufferState.mShared = false; } // Don't put the shared buffer on the free list. if (!mSlots[slot].mBufferState.isShared()) { mCore->mActiveBuffers.erase(slot); // 从mActiveBuffers中删除 mCore->mFreeBuffers.push_back(slot); //加入到mFreeBuffers中 }if (mCore->mBufferReleasedCbEnabled) { listener = mCore->mConnectedProducerListener; / 设置listener } BQ_LOGV("releaseBuffer: releasing slot %d", slot); // 唤醒等待的线程 mCore->mDequeueCondition.notify_all(); VALIDATE_CONSISTENCY(); } // Autolock scope// Call back without lock held if (listener != nullptr) { listener->onBufferReleased(); //通知producer }return NO_ERROR; }

releaseBuffer方法的流程相对简单:
  • slot就是需要释放的BufferSlot的序号;
  • Buffer的FrameNumber变了,可能Buffer已经重新分配,这个是不用管;
  • 只能释放acquire状态的buffer序号,释放后是Buffer放会mFreeBuffers中;
  • releaseFence,从consumer那边传过来,producer可以dequeue mFreeBuffers中的buffer,但是只有releaseFence发信号出来后,consumer才真正用完,producer才可以写;
  • 最后通过listener通知producer。

2.5 ProducerListener是怎样工作的?
在前面的讲解中,有几处都有出现 listener->onBufferReleased() ,意思是通知producer有buffer释放了。这个listener是在哪里设置的?onBufferReleased又做了哪些工作呢?接下来分析
定义:/frameworks/native/libs/gui/include/gui/IProducerListener.h
首先我们看到这张类图:
Android|Android 12(S) 图形显示系统 - BufferQueue的工作流程(十一)
文章图片

1. 在BufferQueueCore中有成员 sp mConnectedProducerListener,它就是用来处理onBufferReleased事件的;
2. mConnectedProducerListener是在哪里被设置的呢?答案是 BufferQueueProducer::connect;
3. 根据调用栈来追踪:
11-13 01:20:19.38829553013 E BufferQueueProducer: stackdump:#00 pc 0005e667/system/lib/libgui.so (android::BufferQueueProducer::connect(android::sp const&, int, bool, android::IGraphicBufferProducer::QueueBufferOutput*)+1018) 11-13 01:20:19.38829553013 E BufferQueueProducer: stackdump:#01 pc 0006ee41/system/lib/libgui.so (android::BBQBufferQueueProducer::connect(android::sp const&, int, bool, android::IGraphicBufferProducer::QueueBufferOutput*)+176) 11-13 01:20:19.38829553013 E BufferQueueProducer: stackdump:#02 pc 000a268b/system/lib/libgui.so (android::Surface::connect(int, android::sp const&, bool)+138) 11-13 01:20:19.38829553013 E BufferQueueProducer: stackdump:#03 pc 0009dd61/system/lib/libgui.so (android::Surface::hook_perform(ANativeWindow*, int, ...)+128)

4. 可以肯定,是执行 native_window_api_connect() 一路走下来的。沿着这条路,看看listener是哪里产生的呢?
看起来是这个位置:
int Surface::connect(int api) { static sp listener = new StubProducerListener(); return connect(api, listener); }

5. 不过,奇怪的事情发生了,StubProducerListener看起来没有任何操作,乖乖(这一点还是很奇怪的)
6. BLASTBufferQueue中还有使用AsyncProducerListener做一层封装,实现异步处理;
class StubProducerListener : public BnProducerListener { public: virtual ~StubProducerListener(); virtual void onBufferReleased() {} virtual bool needsReleaseNotify() { return false; } };

Android|Android 12(S) 图形显示系统 - BufferQueue的工作流程(十一)
文章图片

三、小结 BufferQueue的运作流程到这里就算讲完了。生产者做了什么事情?消费者做了什么事情?图形缓存是怎样流转的?状态是怎样变化的?在几篇文章中基本上都有做了或简单或详细的介绍。通过BufferQueue的几篇文章,帮助自己建立起基本的逻辑框架,为我们后续研究和分析问题奠定基础。
【Android|Android 12(S) 图形显示系统 - BufferQueue的工作流程(十一)】

    推荐阅读