Android|Android 12(S) 图形显示系统 - Surface 一点补充知识（十二）
必读：
Android 12(S) 图形显示系统 - 开篇
一、前言 因为个人工作主要是Android多媒体播放的内容，在工作中查看源码或设计程序经常会遇到调用API：
static inline int native_window_api_connect(struct ANativeWindow* window, int api)
static inline int native_window_api_disconnect(struct ANativeWindow* window, int api)所以也一直好奇这两个方法都做了什么事情？这篇文章就来一探究竟。
二、native_window_api_connect 解析 Android系统中，开始播放视频并设置Surface后，都会做一次 connectToSurface 的操作，比如MediaCodec中，在初始化阶段setSurface后就会调用方法：
status_t MediaCodec::connectToSurface(const sp &surface) {
...
err = nativeWindowConnect(surface.get(), "connectToSurface");
...
}这里就是去调用了 /frameworks/av/media/libstagefright/SurfaceUtils.cpp中的方法：
status_t nativeWindowConnect(ANativeWindow *surface, const char *reason) {
ALOGD("connecting to surface %p, reason %s", surface, reason);
status_t err = native_window_api_connect(surface, NATIVE_WINDOW_API_MEDIA);
ALOGE_IF(err != OK, "Failed to connect to surface %p, err %d", surface, err);
return err;
}是不是看到了 native_window_api_connect ，其中参数 NATIVE_WINDOW_API_MEDIA 表明 video decoder会作为生产者来生成buffer数据。
再接着往下走，看看到底做了什么？
大体的调用流程如下：
/frameworks/native/libs/gui/Surface.cpp
/frameworks/native/libs/gui/BufferQueueProducer.cpp
>>> static inline int native_window_api_connect(struct ANativeWindow* window, int api)
>>> int Surface::hook_perform(ANativeWindow* window, int operation, ...)
>>> int Surface::perform(int operation, va_list args)
>>> int Surface::dispatchConnect(va_list args)
>>> int Surface::connect(int api)
>>> int Surface::connect(int api, const sp
>>> int Surface::connect(int api, const sp
>>> status_t BufferQueueProducer::connect(const sp
int api, bool producerControlledByApp, QueueBufferOutput *output)
文章图片
最终进入到了BufferQueueProducer::connect函数中，看起来这里应该就是做具体事情的地方了
老规矩，看源码：
status_t BufferQueueProducer::connect(const sp& listener,
int api, bool producerControlledByApp, QueueBufferOutput *output) {
ATRACE_CALL();
std::lock_guard lock(mCore->mMutex);
mConsumerName = mCore->mConsumerName;
// 消费者名字
BQ_LOGV("connect: api=%d producerControlledByApp=%s", api,
producerControlledByApp ? "true" : "false");
// 对一些条件进行判断，必要时直接返回return
if (mCore->mIsAbandoned) {
BQ_LOGE("connect: BufferQueue has been abandoned");
return NO_INIT;
}if (mCore->mConsumerListener == nullptr) {
BQ_LOGE("connect: BufferQueue has no consumer");
return NO_INIT;
}if (output == nullptr) {
BQ_LOGE("connect: output was NULL");
return BAD_VALUE;
}if (mCore->mConnectedApi != BufferQueueCore::NO_CONNECTED_API) {
BQ_LOGE("connect: already connected (cur=%d req=%d)",
mCore->mConnectedApi, api);
return BAD_VALUE;
}int delta = mCore->getMaxBufferCountLocked(mCore->mAsyncMode,
mDequeueTimeout < 0 ?
mCore->mConsumerControlledByApp && producerControlledByApp : false,
mCore->mMaxBufferCount) -
mCore->getMaxBufferCountLocked();
if (!mCore->adjustAvailableSlotsLocked(delta)) {
BQ_LOGE("connect: BufferQueue failed to adjust the number of available "
"slots. Delta = %d", delta);
return BAD_VALUE;
}int status = NO_ERROR;
switch (api) {
case NATIVE_WINDOW_API_EGL:
case NATIVE_WINDOW_API_CPU:
case NATIVE_WINDOW_API_MEDIA:
case NATIVE_WINDOW_API_CAMERA:
mCore->mConnectedApi = api;
output->width = mCore->mDefaultWidth;
output->height = mCore->mDefaultHeight;
output->transformHint = mCore->mTransformHintInUse = mCore->mTransformHint;
output->numPendingBuffers =
static_cast(mCore->mQueue.size());
output->nextFrameNumber = mCore->mFrameCounter + 1;
output->bufferReplaced = false;
output->maxBufferCount = mCore->mMaxBufferCount;
if (listener != nullptr) {
// Set up a death notification so that we can disconnect
// automatically if the remote producer dies
#ifndef NO_BINDER
if (IInterface::asBinder(listener)->remoteBinder() != nullptr) {
status = IInterface::asBinder(listener)->linkToDeath(
static_cast(this));
if (status != NO_ERROR) {
BQ_LOGE("connect: linkToDeath failed: %s (%d)",
strerror(-status), status);
}
mCore->mLinkedToDeath = listener;
}
#endif
mCore->mConnectedProducerListener = listener;
// 设置producer listener
mCore->mBufferReleasedCbEnabled = listener->needsReleaseNotify();
}
break;
default:
BQ_LOGE("connect: unknown API %d", api);
status = BAD_VALUE;
break;
}
mCore->mConnectedPid = BufferQueueThreadState::getCallingPid();
mCore->mBufferHasBeenQueued = false;
mCore->mDequeueBufferCannotBlock = false;
mCore->mQueueBufferCanDrop = false;
mCore->mLegacyBufferDrop = true;
if (mCore->mConsumerControlledByApp && producerControlledByApp) {
mCore->mDequeueBufferCannotBlock = mDequeueTimeout < 0;
mCore->mQueueBufferCanDrop = mDequeueTimeout <= 0;
}mCore->mAllowAllocation = true;
// 允许分配 graphic buffer
VALIDATE_CONSISTENCY();
return status;
} 我才疏学浅，按我理解，就是完成了一些初始化的操作，貌似也没啥了，这之后应该produder就可以 dequeue buffer 了
另外一点，这里有设置producer listener，之前文章中也讲过，貌似也没啥作用(也许我错了)
mCore->mConnectedProducerListener = listener;
mCore->mBufferReleasedCbEnabled = listener->needsReleaseNotify();
文章图片
三、native_window_api_disconnect 解析 disconnect的调用流程和connect的流程类似。
先看 Surface::disconnect 中做了啥
int Surface::disconnect(int api, IGraphicBufferProducer::DisconnectMode mode) {
ATRACE_CALL();
ALOGV("Surface::disconnect");
Mutex::Autolock lock(mMutex);
mRemovedBuffers.clear();
mSharedBufferSlot = BufferItem::INVALID_BUFFER_SLOT;
mSharedBufferHasBeenQueued = false;
freeAllBuffers();
int err = mGraphicBufferProducer->disconnect(api, mode);
if (!err) {
mReqFormat = 0;
mReqWidth = 0;
mReqHeight = 0;
mReqUsage = 0;
mCrop.clear();
mScalingMode = NATIVE_WINDOW_SCALING_MODE_FREEZE;
mTransform = 0;
mStickyTransform = 0;
mAutoPrerotation = false;
mEnableFrameTimestamps = false;
mMaxBufferCount = NUM_BUFFER_SLOTS;
if (api == NATIVE_WINDOW_API_CPU) {
mConnectedToCpu = false;
}
}
return err;
}看起来主要是把reset一些变量和release一些资源，看到调用
freeAllBuffers();
int err = mGraphicBufferProducer->disconnect(api, mode)freeAllBuffers是把mSlots里面的元素都置为 nullptr
void Surface::freeAllBuffers() {
if (!mDequeuedSlots.empty()) {
ALOGE("%s: %zu buffers were freed while being dequeued!",
__FUNCTION__, mDequeuedSlots.size());
}
for (int i = 0;
i < NUM_BUFFER_SLOTS;
i++) {
mSlots[i].buffer = nullptr;
}
}本文作者@二的次方2022-03-24 发布于博客园
BufferQueueProducer::disconnect源码如下
status_t BufferQueueProducer::disconnect(int api, DisconnectMode mode) {
ATRACE_CALL();
BQ_LOGV("disconnect: api %d", api);
int status = NO_ERROR;
sp listener;
{ // Autolock scope
std::unique_lock lock(mCore->mMutex);
if (mode == DisconnectMode::AllLocal) {
if (BufferQueueThreadState::getCallingPid() != mCore->mConnectedPid) {
return NO_ERROR;
}
api = BufferQueueCore::CURRENTLY_CONNECTED_API;
}mCore->waitWhileAllocatingLocked(lock);
if (mCore->mIsAbandoned) {
// It's not really an error to disconnect after the surface has
// been abandoned;
it should just be a no-op.
return NO_ERROR;
}if (api == BufferQueueCore::CURRENTLY_CONNECTED_API) {
if (mCore->mConnectedApi == NATIVE_WINDOW_API_MEDIA) {
ALOGD("About to force-disconnect API_MEDIA, mode=%d", mode);
}
api = mCore->mConnectedApi;
// If we're asked to disconnect the currently connected api but
// nobody is connected, it's not really an error.
if (api == BufferQueueCore::NO_CONNECTED_API) {
return NO_ERROR;
}
}switch (api) {
case NATIVE_WINDOW_API_EGL:
case NATIVE_WINDOW_API_CPU:
case NATIVE_WINDOW_API_MEDIA:
case NATIVE_WINDOW_API_CAMERA:
if (mCore->mConnectedApi == api) {
mCore->freeAllBuffersLocked();
#ifndef NO_BINDER
// Remove our death notification callback if we have one
if (mCore->mLinkedToDeath != nullptr) {
sp token =
IInterface::asBinder(mCore->mLinkedToDeath);
// This can fail if we're here because of the death
// notification, but we just ignore it
token->unlinkToDeath(
static_cast(this));
}
#endif
mCore->mSharedBufferSlot =
BufferQueueCore::INVALID_BUFFER_SLOT;
mCore->mLinkedToDeath = nullptr;
mCore->mConnectedProducerListener = nullptr;
mCore->mConnectedApi = BufferQueueCore::NO_CONNECTED_API;
mCore->mConnectedPid = -1;
mCore->mSidebandStream.clear();
mCore->mDequeueCondition.notify_all();
mCore->mAutoPrerotation = false;
listener = mCore->mConsumerListener;
} else if (mCore->mConnectedApi == BufferQueueCore::NO_CONNECTED_API) {
BQ_LOGE("disconnect: not connected (req=%d)", api);
status = NO_INIT;
} else {
BQ_LOGE("disconnect: still connected to another API "
"(cur=%d req=%d)", mCore->mConnectedApi, api);
status = BAD_VALUE;
}
break;
default:
BQ_LOGE("disconnect: unknown API %d", api);
status = BAD_VALUE;
break;
}
} // Autolock scope// Call back without lock held
if (listener != nullptr) {
listener->onBuffersReleased();
listener->onDisconnect();
}return status;
} 看样子也是reset一些变量和release资源。
调用了mCore->freeAllBuffersLocked()
void BufferQueueCore::freeAllBuffersLocked() {
for (int s : mFreeSlots) {
clearBufferSlotLocked(s);
}for (int s : mFreeBuffers) {
mFreeSlots.insert(s);
clearBufferSlotLocked(s);
}
mFreeBuffers.clear();
for (int s : mActiveBuffers) {
mFreeSlots.insert(s);
clearBufferSlotLocked(s);
}
mActiveBuffers.clear();
for (auto& b : mQueue) {
b.mIsStale = true;
b.mAcquireCalled = false;
}VALIDATE_CONSISTENCY();
}【Android|Android 12(S) 图形显示系统 - Surface 一点补充知识（十二）】通知了消费者 listener->onBuffersReleased()and listener->onDisconnect()
消费者响应 onBuffersReleased 也是去free buffer
void ConsumerBase::onBuffersReleased() {
Mutex::Autolock lock(mMutex);
CB_LOGV("onBuffersReleased");
if (mAbandoned) {
// Nothing to do if we're already abandoned.
return;
}uint64_t mask = 0;
mConsumer->getReleasedBuffers(&mask);
for (int i = 0;
i < BufferQueue::NUM_BUFFER_SLOTS;
i++) {
if (mask & (1ULL << i)) {
freeBufferLocked(i);
}
}
}消费者响应 onDisconnect
void BLASTBufferItemConsumer::onDisconnect() {
Mutex::Autolock lock(mMutex);
mPreviouslyConnected = mCurrentlyConnected;
mCurrentlyConnected = false;
if (mPreviouslyConnected) {
mDisconnectEvents.push(mCurrentFrameNumber);
}
mFrameEventHistory.onDisconnect();
}到此，就讲完了，感觉好乏味...，没有什么实质的东西
? Android 12 Google将buffer queue组件从SurfaceFlinger端移动到了客户端（旧版本是在BufferQueueLayer中去createBufferQueue的）。 buffer queue组件的创建和初始化也放在BLASTBufferQueue中。通过类名可以看出BLASTBufferQueue更像是buffer queue组件的一层封装或装饰。 ? 通过前面系列文章的分析，可以看到，整个生产消费模型都在客户端，图形缓冲区的出队、入队、获取等操作都在客户端完成，预示着producer -- buffer queue -- consumer 间的通信都变成了本地通信。 ? BLASTBufferQueue需要通过事务Transaction来向SurfaceFlinger端提交Buffer与图层的属性。 ? 如本文讲的 disconnect event 在12平台就无法传递到SurfaceFlinger了。如要传递信息，就要使用 Transaction 传递过去。
不过我还是有个疑问：disconnect过程中，可以看到producer/consumer/surface都有去free buffer，此时，为GraphicBuffer分配的内存真的就释放了吗？
(mSlots[slot].mGraphicBuffer.clear() ormSlots[slotIndex].mGraphicBuffer = nullptr)
文章图片
四、小结 感觉没啥说的了，就此结束吧
文章图片
推荐阅读
- python统计数据画概率曲线_利用Python实现并解释概率图形
- android|android 利用数据库实现历史搜索记录功能
- Android|android属性之noHistory
- android|Android历史版本
- android|android studio如何复制错误信息
- Android|TabLayout+ViewPager2的联合使用
- android|android UI绘制加减号按钮
- Android面试（怎样备战2022春招（这份高级工程师整理的《Android八大模块进阶指南》拿去吧！））
- Android|薪资倒挂（行业内卷？2022年最新Android学习路线整理分享，带你干倒世界卷王）
- Android实现绚丽的自定义进度条