同时使用多个摄像头流

注意：本页面引用 Camera2 软件包。除非您的应用需要 Camera2 的特定低级功能，否则我们建议使用 CameraX。CameraX 和 Camera2 都支持 Android 5.0（API 级别 21）及更高版本。

相机应用可以同时使用多个帧流。在某些情况下，不同的流甚至需要不同的帧分辨率或像素格式。一些典型的用例包括

视频录制：一个流用于预览，另一个流用于编码并保存到文件。
条形码扫描：一个流用于预览，另一个流用于条形码检测。
计算摄影：一个流用于预览，另一个流用于人脸/场景检测。

处理帧时会产生不小的性能开销，而并行流或流水线处理时，开销会成倍增加。

CPU、GPU 和 DSP 等资源可能会利用框架的再处理功能，但内存等资源将线性增长。

每个请求多个目标

多个摄像头流可以组合成一个CameraCaptureRequest。以下代码片段演示了如何设置一个摄像头会话，其中一个流用于相机预览，另一个流用于图像处理：

Kotlin

val session: CameraCaptureSession = ...  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
val requestTemplate = CameraDevice.TEMPLATE_PREVIEW
val combinedRequest = session.device.createCaptureRequest(requestTemplate)

// Link the Surface targets with the combined request
combinedRequest.addTarget(previewSurface)
combinedRequest.addTarget(imReaderSurface)

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
session.setRepeatingRequest(combinedRequest.build(), null, null)

Java

CameraCaptureSession session = …;  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
        CaptureRequest.Builder combinedRequest = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);

// Link the Surface targets with the combined request
        combinedRequest.addTarget(previewSurface);
        combinedRequest.addTarget(imReaderSurface);

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
        session.setRepeatingRequest(combinedRequest.build(), null, null);

如果您正确配置目标表面，此代码将仅生成满足由 StreamComfigurationMap.GetOutputMinFrameDuration(int, Size) 和 StreamComfigurationMap.GetOutputStallDuration(int, Size) 确定的最小 FPS 的流。实际性能因设备而异，但 Android 保证支持特定的组合，具体取决于三个变量：输出类型、输出大小和硬件级别。

使用不支持的变量组合可能在低帧率下工作；如果不行，将触发其中一个失败回调。 createCaptureSession 的文档描述了哪些是保证能工作的。

输出类型

输出类型是指帧的编码格式。可能的值为 PRIV、YUV、JPEG 和 RAW。 createCaptureSession 的文档描述了它们。

在选择应用的输出类型时，如果目标是最大化兼容性，则使用 ImageFormat.YUV_420_888 进行帧分析，使用 ImageFormat.JPEG 进行静态图像。对于预览和录制场景，您可能会使用 SurfaceView、TextureView、MediaRecorder、MediaCodec 或 RenderScript.Allocation。在这些情况下，请勿指定图像格式。为了兼容性，无论内部实际使用的格式如何，它都将计为 ImageFormat.PRIVATE。要查询设备支持的格式，给定其 CameraCharacteristics，请使用以下代码：

Kotlin

val characteristics: CameraCharacteristics = ...
val supportedFormats = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).outputFormats

Java

CameraCharacteristics characteristics = …;
        int[] supportedFormats = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).getOutputFormats();

输出大小

所有可用的输出大小均由 StreamConfigurationMap.getOutputSizes() 列出，但只有两个与兼容性相关：PREVIEW 和 MAXIMUM。这些大小充当上限。如果 PREVIEW 大小的内容可以工作，则任何小于 PREVIEW 大小的内容也将工作。MAXIMUM 也是如此。 CameraDevice 的文档解释了这些大小。

可用的输出大小取决于所选格式。给定 CameraCharacteristics 和一种格式，您可以像这样查询可用的输出大小：

Kotlin

val characteristics: CameraCharacteristics = ...
val outputFormat: Int = ...  // such as ImageFormat.JPEG
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(outputFormat)

Java

CameraCharacteristics characteristics = …;
        int outputFormat = …;  // such as ImageFormat.JPEG
Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

在相机预览和录制用例中，使用目标类来确定支持的大小。格式将由相机框架本身处理：

Kotlin

val characteristics: CameraCharacteristics = ...
val targetClass: Class <T> = ...  // such as SurfaceView::class.java
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(targetClass)

Java

CameraCharacteristics characteristics = …;
   int outputFormat = …;  // such as ImageFormat.JPEG
   Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

要获取 MAXIMUM 大小，请按面积对输出大小进行排序并返回最大的一个：

Kotlin

fun <T>getMaximumOutputSize(
    characteristics: CameraCharacteristics, targetClass: Class <T>, format: Int? = null):
    Size {
  val config = characteristics.get(
      CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)

  // If image format is provided, use it to determine supported sizes; or else use target class
  val allSizes = if (format == null)
    config.getOutputSizes(targetClass) else config.getOutputSizes(format)
  return allSizes.maxBy { it.height * it.width }
}

Java

 @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getMaximumOutputSize(CameraCharacteristics characteristics,
                                            Class <T> targetClass,
                                            Integer format) {
        StreamConfigurationMap config = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);

        // If image format is provided, use it to determine supported sizes; else use target class
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }
        return Arrays.stream(allSizes).max(Comparator.comparing(s -> s.getHeight() * s.getWidth())).get();
    }

PREVIEW 指的是与设备屏幕分辨率或 1080p (1920x1080) 最佳匹配的大小，取两者中较小者。宽高比可能与屏幕的宽高比不完全匹配，因此您可能需要对流应用信箱模式或裁剪以全屏显示。要获取正确的预览大小，请将可用输出大小与显示大小进行比较，同时考虑到显示可能已旋转。

以下代码定义了一个辅助类 SmartSize，它将使大小比较变得更容易一些：

Kotlin

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize(width: Int, height: Int) {
    var size = Size(width, height)
    var long = max(size.width, size.height)
    var short = min(size.width, size.height)
    override fun toString() = "SmartSize(${long}x${short})"
}

/** Standard High Definition size for pictures and video */
val SIZE_1080P: SmartSize = SmartSize(1920, 1080)

/** Returns a [SmartSize] object for the given [Display] */
fun getDisplaySmartSize(display: Display): SmartSize {
    val outPoint = Point()
    display.getRealSize(outPoint)
    return SmartSize(outPoint.x, outPoint.y)
}

/**
 * Returns the largest available PREVIEW size. For more information, see:
 * https://d.android.com/reference/android/hardware/camera2/CameraDevice
 */
fun <T>getPreviewOutputSize(
        display: Display,
        characteristics: CameraCharacteristics,
        targetClass: Class <T>,
        format: Int? = null
): Size {

    // Find which is smaller: screen or 1080p
    val screenSize = getDisplaySmartSize(display)
    val hdScreen = screenSize.long >= SIZE_1080P.long || screenSize.short >= SIZE_1080P.short
    val maxSize = if (hdScreen) SIZE_1080P else screenSize

    // If image format is provided, use it to determine supported sizes; else use target class
    val config = characteristics.get(
            CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)!!
    if (format == null)
        assert(StreamConfigurationMap.isOutputSupportedFor(targetClass))
    else
        assert(config.isOutputSupportedFor(format))
    val allSizes = if (format == null)
        config.getOutputSizes(targetClass) else config.getOutputSizes(format)

    // Get available sizes and sort them by area from largest to smallest
    val validSizes = allSizes
            .sortedWith(compareBy { it.height * it.width })
            .map { SmartSize(it.width, it.height) }.reversed()

    // Then, get the largest output size that is smaller or equal than our max size
    return validSizes.first { it.long <= maxSize.long && it.short <= maxSize.short }.size
}

Java

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
    class SmartSize {
        Size size;
        double longSize;
        double shortSize;

        public SmartSize(Integer width, Integer height) {
            size = new Size(width, height);
            longSize = max(size.getWidth(), size.getHeight());
            shortSize = min(size.getWidth(), size.getHeight());
        }

        @Override
        public String toString() {
            return String.format("SmartSize(%sx%s)", longSize, shortSize);
        }
    }

    /** Standard High Definition size for pictures and video */
    SmartSize SIZE_1080P = new SmartSize(1920, 1080);

    /** Returns a [SmartSize] object for the given [Display] */
    SmartSize getDisplaySmartSize(Display display) {
        Point outPoint = new Point();
        display.getRealSize(outPoint);
        return new SmartSize(outPoint.x, outPoint.y);
    }

    /**
     * Returns the largest available PREVIEW size. For more information, see:
     * https://d.android.com/reference/android/hardware/camera2/CameraDevice
     */
    @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getPreviewOutputSize(
            Display display,
            CameraCharacteristics characteristics,
            Class <T> targetClass,
            Integer format
    ){

        // Find which is smaller: screen or 1080p
        SmartSize screenSize = getDisplaySmartSize(display);
        boolean hdScreen = screenSize.longSize >= SIZE_1080P.longSize || screenSize.shortSize >= SIZE_1080P.shortSize;
        SmartSize maxSize;
        if (hdScreen) {
            maxSize = SIZE_1080P;
        } else {
            maxSize = screenSize;
        }

        // If image format is provided, use it to determine supported sizes; else use target class
        StreamConfigurationMap config = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
        if (format == null)
            assert(StreamConfigurationMap.isOutputSupportedFor(targetClass));
        else
            assert(config.isOutputSupportedFor(format));
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }

        // Get available sizes and sort them by area from largest to smallest
        List <Size> sortedSizes = Arrays.asList(allSizes);
        List <SmartSize> validSizes =
                sortedSizes.stream()
                        .sorted(Comparator.comparing(s -> s.getHeight() * s.getWidth()))
                        .map(s -> new SmartSize(s.getWidth(), s.getHeight()))
                        .sorted(Collections.reverseOrder()).collect(Collectors.toList());

        // Then, get the largest output size that is smaller or equal than our max size
        return validSizes.stream()
                .filter(s -> s.longSize <= maxSize.longSize && s.shortSize <= maxSize.shortSize)
                .findFirst().get().size;
    }

检查支持的硬件级别

要在运行时确定可用功能，请使用 CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL 检查支持的硬件级别。

使用 CameraCharacteristics 对象，您可以通过单个语句检索硬件级别：

Kotlin

val characteristics: CameraCharacteristics = ...

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
val hardwareLevel = characteristics.get(
        CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL)

Java

CameraCharacteristics characteristics = ...;

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
Integer hardwareLevel = characteristics.get(
                CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);

整合所有部分

结合输出类型、输出大小和硬件级别，您可以确定哪些流组合是有效的。下表是 LEGACY 硬件级别的 CameraDevice 支持的配置快照。

目标 1		目标 2		目标 3		示例用例
类型	最大尺寸	类型	最大尺寸	类型	最大尺寸	示例用例
`PRIV`	`MAXIMUM`					简单预览、GPU 视频处理或无预览视频录制。
`JPEG`	`MAXIMUM`					无取景器静态图像捕捉。
`YUV`	`MAXIMUM`					应用内视频/图像处理。
`PRIV`	`PREVIEW`	`JPEG`	`MAXIMUM`			标准静态图像。
`YUV`	`PREVIEW`	`JPEG`	`MAXIMUM`			应用内处理加静态捕捉。
`PRIV`	`PREVIEW`	`PRIV`	`PREVIEW`			标准录制。
`PRIV`	`PREVIEW`	`YUV`	`PREVIEW`			预览加应用内处理。
`PRIV`	`PREVIEW`	`YUV`	`PREVIEW`			预览加应用内处理。
`PRIV`	`PREVIEW`	`YUV`	`PREVIEW`	`JPEG`	`MAXIMUM`	静态捕捉加应用内处理。

LEGACY 硬件级别针对最低公分母设备。此表显示，所有支持 Camera2（API 级别 21 及更高版本）的设备都可以使用正确的配置输出最多三个同步流，并且如果限制性能的开销（如内存、CPU 或热量限制）不是太大。

您的应用还需要配置目标输出缓冲区。例如，要定位具有 LEGACY 硬件级别的设备，您可以设置两个目标输出表面，一个使用 ImageFormat.PRIVATE，另一个使用 ImageFormat.YUV_420_888。在使用 PREVIEW 大小时，这是一种受支持的组合。使用本主题中先前定义的函数，获取相机 ID 所需的预览大小需要以下代码：

Kotlin

val characteristics: CameraCharacteristics = ...
val context = this as Context  // assuming you are inside of an activity

val surfaceViewSize = getPreviewOutputSize(
    context, characteristics, SurfaceView::class.java)
val imageReaderSize = getPreviewOutputSize(
    context, characteristics, ImageReader::class.java, format = ImageFormat.YUV_420_888)

Java

CameraCharacteristics characteristics = ...;
        Context context = this; // assuming you are inside of an activity

        Size surfaceViewSize = getPreviewOutputSize(
                context, characteristics, SurfaceView.class);
        Size imageReaderSize = getPreviewOutputSize(
                context, characteristics, ImageReader.class, format = ImageFormat.YUV_420_888);

它需要使用提供的回调等待 SurfaceView 准备就绪：

Kotlin

val surfaceView = findViewById <SurfaceView>(...)
surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
  override fun surfaceCreated(holder: SurfaceHolder) {
    // You do not need to specify image format, and it will be considered of type PRIV
    // Surface is now ready and you could use it as an output target for CameraSession
  }
  ...
})

Java

SurfaceView surfaceView = findViewById <SurfaceView>(...);

surfaceView.getHolder().addCallback(new SurfaceHolder.Callback() {
            @Override
            public void surfaceCreated(@NonNull SurfaceHolder surfaceHolder) {
                // You do not need to specify image format, and it will be considered of type PRIV
                // Surface is now ready and you could use it as an output target for CameraSession
            }
            ...
        });

您可以通过调用 SurfaceHolder.setFixedSize() 强制 SurfaceView 匹配相机输出大小，或者您可以采用类似于 GitHub 相机示例的通用模块中的 AutoFitSurfaceView 的方法，该方法设置一个绝对大小，同时考虑了宽高比和可用空间，并在触发 Activity 更改时自动调整。

从 ImageReader 设置具有所需格式的另一个表面更容易，因为无需等待回调：

Kotlin

val frameBufferCount = 3  // just an example, depends on your usage of ImageReader
val imageReader = ImageReader.newInstance(
    imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
    frameBufferCount)

Java

int frameBufferCount = 3;  // just an example, depends on your usage of ImageReader
ImageReader imageReader = ImageReader.newInstance(
                imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
                frameBufferCount);

当使用像 ImageReader 这样的阻塞目标缓冲区时，在使用完帧后将其丢弃：

Kotlin

imageReader.setOnImageAvailableListener({
  val frame =  it.acquireNextImage()
  // Do something with "frame" here
  it.close()
}, null)

Java

imageReader.setOnImageAvailableListener(listener -> {
            Image frame = listener.acquireNextImage();
            // Do something with "frame" here
            listener.close();
        }, null);

LEGACY 硬件级别针对最低公分母设备。您可以添加条件分支，并在具有 LIMITED 硬件级别的设备中使用 RECORD 大小作为其中一个输出目标表面，甚至对于具有 FULL 硬件级别的设备，可以将其增加到 MAXIMUM 大小。

同时使用多个摄像头流 使用收藏夹整理内容 根据您的偏好保存和分类内容。

每个请求多个目标

Kotlin

Java

输出类型

Kotlin

Java

输出大小

Kotlin

Java

Kotlin

Java

Kotlin

Java

Kotlin

Java

检查支持的硬件级别

Kotlin

Java

整合所有部分

Kotlin

Java

Kotlin

Java

Kotlin

Java

Kotlin

Java

同时使用多个摄像头流