Flutter FlutterImageView

在 Flutter 中，Android 下使用 FlutterView 展示 Flutter UI。FlutterView 支持三种展示模式：FlutterTextureView、FlutterSurfaceView、FlutterImageView。

Flutter 支持 PlatformView 能力，能够将原生平台视图，嵌入到 Flutter UI 中展示。PlatformView 由分为多种模式，其中一种模式是 Hybrid composition 模式，在这种模式下，FlutterView 需要使用 FlutterImageView 进行配套展示。

在日常开发中，我们通常使用 FlutterTextureView 或 FlutterSurfaceView。当 Flutter UI 中出现嵌入原生视图时，FlutterView 会自动切换到 FlutterImageView 模式下，当嵌入原生视图从 Flutter UI 中消失后，FlutterView 又会自动切换至原来的模式。

在 Hybrid composition 渲染模式下，FlutterView 为了展示嵌入的原生视图，会采取一种类似”汉堡包“的多层级视图：

• 最底层 FlutterImageView：用于替换原来的 FlutterTextureView/FlutterSurfaceView。显示 Flutter UI 底图。所谓“底图”，指的是嵌入原生视图本身会占据一定空间，Flutter 会“偷懒”，省去这部分被挡住的绘制。
• 中间层 FlutterMutatorView：FlutterMutatorView 是一种 FrameLayout，原生视图将加入其中。这也是 Hybrid composition 模式的特色，即原生视图是直接挂载到 Android 原生视图树上的，与 Android 视图系统天然融合，相较于最早的 VirtualDisplay 模式，少了很多兼容性问题。
• 最顶层 FlutterImageView：在 Flutter 布局中，嵌入原生视图之上，很可能还有 Flutter 视图覆盖（Overlay），因此在 FlutterView 中，在 FlutterMutatorView 上层会有一个或者多个 FlutterImageView 来展示覆盖在嵌入原生视图之上的 Flutter UI。之所以数量不定，是由 Hybrid composition 的算法故意如此设计的。

总结 FlutterImageView 的功能：

1. Flutter Android Hybrid composition PlatformView 模式下的 FlutterView 渲染模式
2. 用于替换原来的 FlutterTextureView/FlutterSurfaceView，渲染底层 Flutter UI
3. 渲染覆盖在嵌入原生视图之上的 Flutter UI

注：FlutterImageView 不是 Android 的 ImageView，不要搞混了

核心成员

FlutterImageView 下的核心成员：

• imageReader：捕获 Flutter UI 的输出
• currentImage 和 currentBitmap：存储渲染后的帧数据
• Surface：向 FlutterRenderer 提供一个可渲染的 Surface
• Canvas：负责实际的绘制操作

核心工作流程

1. Flutter Engine 将 Flutter UI 渲染到 imageReader
2. imageReader 捕获帧，保存到 Image
3. 从 Image 转换出 Bitmap
4. 将 Bitmap 绘制到 Canvas 中。

具体代码实现：

创建 imageReader 实例

private static ImageReader createImageReader(int width, int height) {
  // ……
  if (android.os.Build.VERSION.SDK_INT >= 29) {
    return ImageReader.newInstance(
        width,
        height,
        PixelFormat.RGBA_8888,
        3,
        HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE | HardwareBuffer.USAGE_GPU_COLOR_OUTPUT);
  } else {
    return ImageReader.newInstance(width, height, PixelFormat.RGBA_8888, 3);
  }
}

做了版本区分，API 29（Android 10）以上走 GPU。这也是为什么说 Android 10 以下版本 Hybrid composition 模式性能下降的原因。走 GPU 的话，帧是从 GPU -> GPU 的传递。如果低版本，需要 GPU -> CPU -> GPU，多了一层内存拷贝的过程。

将 imageReader 绑定到 FlutterRenderer

/**
 * Invoked by the owner of this {@code FlutterImageView} when it wants to begin rendering a
 * Flutter UI to this {@code FlutterImageView}.
 */
@Override
public void attachToRenderer(@NonNull FlutterRenderer flutterRenderer) {
  switch (kind) {
    case background:
      flutterRenderer.swapSurface(imageReader.getSurface());
      break;
    case overlay:
      // Do nothing since the attachment is done by the handler of
      // `FlutterJNI#createOverlaySurface()` in the native side.
      break;
  }
  setAlpha(1.0f);
  this.flutterRenderer = flutterRenderer;
  isAttachedToFlutterRenderer = true;
}

其中，kind 是 FlutterImageView 中的一个枚举，指示 FlutterImageView 是作为底图（background）还是覆盖（overlay）：

public enum SurfaceKind {
  /** Displays the background canvas. */
  background,

  /** Displays the overlay surface canvas. */
  overlay,
}

在上面代码中：

• 如果 FlutterImageView 是 background：需要用自己的 Surface 替换 flutterRenderer 的 Surface
• 如果 FlutterImageView 是 overlay：attachToRenderer 时不需要做什么，它是响应 overlay surface 的绘制流程

处理新视图帧

acquireLatestImage 方法用于读取新的帧，该方法由外部调用，将在后续小节中分析。需要注意的是，拿到新的帧后，会调用 invalidate 触发一次绘制流程，走到接下来的上屏逻辑。

/**
 * Acquires the next image to be drawn to the {@link android.graphics.Canvas}. Returns true if
 * there's an image available in the queue.
 */
@TargetApi(19)
public boolean acquireLatestImage() {
  if (!isAttachedToFlutterRenderer) {
    return false;
  }
  // 1. `acquireLatestImage()` may return null if no new image is available.
  // 2. There's no guarantee that `onDraw()` is called after `invalidate()`.
  // For example, the device may not produce new frames if it's in sleep mode
  // or some special Android devices so the calls to `invalidate()` queued up
  // until the device produces a new frame.
  // 3. While the engine will also stop producing frames, there is a race condition.
  final Image newImage = imageReader.acquireLatestImage();
  if (newImage != null) {
    // Only close current image after acquiring valid new image
    closeCurrentImage();
    currentImage = newImage;
    invalidate();
  }
  return newImage != null;
}

上屏

FlutterImageView 复写了 View 的 onDraw 方法，将 Bitmap 绘制到 Android 视图上：

@Override
protected void onDraw(Canvas canvas) {
  super.onDraw(canvas);
  if (currentImage != null) {
    updateCurrentBitmap();
  }
  if (currentBitmap != null) {
    canvas.drawBitmap(currentBitmap, 0, 0, null);
  }
}

其中，updateCurrentBitmap 方法用于将 Image 转换为 Bitmap。里面区分 API 29，分为走 GPU 和不走。

至此，FlutterImageView 完成了屏幕展示。

FlutterImageView 模式下的视图更新

当 FlutterView 位于 FlutterImageView 模式下，视图是如何更新的呢？

PlatformViewsController 负责 PlatformView 模式下的相关操作，其中 onEndFrame 负责一帧结束后，更新 FlutterImageView。具体实现如下：

/**
 * Called by {@code FlutterJNI} when the Flutter frame was submitted.
 *
 * <p>This member is not intended for public use, and is only visible for testing.
 */
public void onEndFrame() {
  // 如果 FlutterView 处于 FlutterImageView 模式，但是这一帧里没有嵌原生视图
  // Flutter 会立刻自动将 FlutterImageView 模式切换回原模式
  // If there are no platform views in the current frame,
  // then revert the image view surface and use the previous surface.
  //
  // Otherwise, acquire the latest image.
  if (flutterViewConvertedToImageView && currentFrameUsedPlatformViewIds.isEmpty()) {
    flutterViewConvertedToImageView = false;
    flutterView.revertImageView(
        () -> {
          // Destroy overlay surfaces once the surface reversion is completed.
          finishFrame(false);
        });
    return;
  }
  // Whether the current frame was rendered using ImageReaders.
  //
  // Since the image readers may not have images available at this point,
  // this becomes true if all the required surfaces have images available.
  //
  // This is used to decide if the platform views can be rendered in the current frame.
  // If one of the surfaces doesn't have an image, the frame may be incomplete and must be
  // dropped.
  // For example, a toolbar widget painted by Flutter may not be rendered.
  // 能走到这，说明是 FlutterImageView 模式，且当前帧有嵌原生视图
  // 并且 flutterView（底图那个）成功获取到了新的帧
  final boolean isFrameRenderedUsingImageReaders =
      flutterViewConvertedToImageView && flutterView.acquireLatestImageViewFrame();
  finishFrame(isFrameRenderedUsingImageReaders);
}

注意，上面代码中，在 Flutter Engine 完成一帧帧绘制后，触发 flutterView 去同步了最新的帧。

finishFrame 负责 platformView 和上层的 overlay FlutterImageView 处理，它接收一个参数，表明当前帧是否与 ImageRender 关联：

private void finishFrame(boolean isFrameRenderedUsingImageReaders) {
  //======================
  // 更新当前帧的 Overlays
  //======================
  // 获取覆盖在原生视图上面的 Overlay Flutter UI，根据 HC 模式的算法，可能有多个
  // 所有的 Overlay
  for (int i = 0; i < overlayLayerViews.size(); i++) {
    final int overlayId = overlayLayerViews.keyAt(i);
    // 这些 FlutterImageView 提前由 createOverlaySurface 创建好
    final FlutterImageView overlayView = overlayLayerViews.valueAt(i);
    // 过滤出当前帧所包含的
    if (currentFrameUsedOverlayLayerIds.contains(overlayId)) {
	  // 将这些 overlayView 关联到 FlutterRenderer
	  // 但因为他们是 overlay 类型，实际什么都没做
      flutterView.attachOverlaySurfaceToRender(overlayView);
      // 请求一帧
      final boolean didAcquireOverlaySurfaceImage = overlayView.acquireLatestImage();
      isFrameRenderedUsingImageReaders &= didAcquireOverlaySurfaceImage;
    } else {
	  // 如果当前帧里不包含这个 overlay
      // If the background surface isn't rendered by the image view, then the
      // overlay surfaces can be detached from the rendered.
      // This releases resources used by the ImageReader.
      // 如果 FlutterView 模式已经不是 ImageView，将 OverView 与 Renderer 解绑
      if (!flutterViewConvertedToImageView) {
        overlayView.detachFromRenderer();
      }
      // 隐藏掉不再当前帧的 overlay
      // Hide overlay surfaces that aren't rendered in the current frame.
      overlayView.setVisibility(View.GONE);
    }
  }

  // 展示当前帧包含的 PlatformView
  for (int i = 0; i < platformViewParent.size(); i++) {
    final int viewId = platformViewParent.keyAt(i);
    final View parentView = platformViewParent.get(viewId);

    // This should only show platform views that are rendered in this frame and either:
    //  1. Surface has images available in this frame or,
    //  2. Surface does not have images available in this frame because the render surface should
    // not be an ImageView.
    //
    // The platform view is appended to a mutator view.
    //
    // Otherwise, hide the platform view, but don't remove it from the view hierarchy yet as
    // they are removed when the framework diposes the platform view widget.
    if (currentFrameUsedPlatformViewIds.contains(viewId)
        && (isFrameRenderedUsingImageReaders || !synchronizeToNativeViewHierarchy)) {
      parentView.setVisibility(View.VISIBLE);
    } else {
      parentView.setVisibility(View.GONE);
    }
  }
}

销毁生命周期

FlutterImageView 内部销毁方法

detachFromRenderer：从 FlutterRenderer 中解绑，同时会清空 Bitmap，并触发绘制，变成空白并且透明:

/**
 * Invoked by the owner of this {@code FlutterImageView} when it no longer wants to render a
 * Flutter UI to this {@code FlutterImageView}.
 */
public void detachFromRenderer() {
  if (!isAttachedToFlutterRenderer) {
    return;
  }
  // 设置为透明
  setAlpha(0.0f);
  // Drop the latest image as it shouldn't render this image if this view is
  // attached to the renderer again.
  acquireLatestImage();
  // Clear drawings.
  currentBitmap = null;

  // Close and clear the current image if any.
  closeCurrentImage();
  invalidate();
  isAttachedToFlutterRenderer = false;
}

public void pause() {
  // Not supported.
}

closeImageReader：关闭 ImageReader

/**
 * Closes the image reader associated with the current {@code FlutterImageView}.
 *
 * <p>Once the image reader is closed, calling {@code acquireLatestImage} will result in an {@code
 * IllegalStateException}.
 */
public void closeImageReader() {
  imageReader.close();
}

从注解中看出，imageReader 关闭之后，就不能调用 acquireLatestImage 了。

closeCurrentImage：关闭 Image：

private void closeCurrentImage() {
  // Close and clear the current image if any.
  if (currentImage != null) {
    currentImage.close();
    currentImage = null;
  }
}

外部销毁生命周期

FlutterView detachFromFlutterEngine

flutterEngine.getPlatformViewsController().detachFromView();
//...
flutterRenderer.stopRenderingToSurface();
//...
renderSurface.detachFromRenderer();
if (flutterImageView != null) {
  flutterImageView.closeImageReader();
  flutterImageView = null;
}

FlutterView revertImageView

将 FlutterImageView 模式转换为之前的模式。

PlatformViewsController finishFrame

当前帧中不包含的 overlay，会 detachFromRenderer

PlatformViewsController destroyOverlaySurfaces、removeOverlaySurfaces

销毁 overlay ImageViews。

PlatformViewsController detachFromView

/**
 * This {@code PlatformViewController} and its {@code FlutterEngine} are no longer attached to an
 * Android {@code View} that renders a Flutter UI.
 *
 * <p>All platform views controlled by this {@code PlatformViewController} will be detached from
 * the previously attached {@code View}.
 */
public void detachFromView() {
  destroyOverlaySurfaces();
  removeOverlaySurfaces();
  this.flutterView = null;
  flutterViewConvertedToImageView = false;

  // Inform all existing platform views that they are no longer associated with
  // a Flutter View.
  for (VirtualDisplayController controller : vdControllers.values()) {
    controller.onFlutterViewDetached();
  }
}

本文作者：Maeiee

本文链接：Flutter FlutterImageView

版权声明：如无特别声明，本文即为原创文章，版权归 Maeiee 所有，未经允许不得转载！

喜欢我文章的朋友请随缘打赏，鼓励我创作更多更好的作品！