同时使用多个摄像头流

注意：此页面指的是 Camera2 软件包。除非您的应用需要 Camera2 的特定低级别功能，否则我们建议使用 CameraX。CameraX 和 Camera2 都支持 Android 5.0（API 级别 21）及更高版本。

相机应用程序可以同时使用多个帧流。在某些情况下，不同的流甚至需要不同的帧分辨率或像素格式。一些典型的用例包括

视频录制：一个流用于预览，另一个流用于编码并保存到文件中。
条形码扫描：一个流用于预览，另一个流用于条形码检测。
计算摄影：一个流用于预览，另一个流用于人脸/场景检测。

处理帧时存在非平凡的性能成本，并且在进行并行流或流水线处理时，该成本会成倍增加。

CPU、GPU 和 DSP 等资源可能能够利用框架的重处理功能，但内存等资源将线性增长。

每个请求的多个目标

多个相机流可以组合成单个CameraCaptureRequest。以下代码片段说明如何设置一个相机会话，该会话包含一个用于相机预览的流和另一个用于图像处理的流

Kotlin

val session: CameraCaptureSession = ...  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
val requestTemplate = CameraDevice.TEMPLATE_PREVIEW
val combinedRequest = session.device.createCaptureRequest(requestTemplate)

// Link the Surface targets with the combined request
combinedRequest.addTarget(previewSurface)
combinedRequest.addTarget(imReaderSurface)

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
session.setRepeatingRequest(combinedRequest.build(), null, null)

Java

CameraCaptureSession session = …;  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
        CaptureRequest.Builder combinedRequest = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);

// Link the Surface targets with the combined request
        combinedRequest.addTarget(previewSurface);
        combinedRequest.addTarget(imReaderSurface);

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
        session.setRepeatingRequest(combinedRequest.build(), null, null);

如果正确配置目标表面，此代码只会生成满足由StreamComfigurationMap.GetOutputMinFrameDuration(int, Size)和StreamComfigurationMap.GetOutputStallDuration(int, Size)确定的最小 FPS 的流。不过，实际性能因设备而异，尽管 Android 提供了一些保证，可以根据三个变量支持特定的组合：输出类型、输出大小和硬件级别。

使用不受支持的变量组合可能会以低帧率工作；如果不行，它将触发一个失败回调。 createCaptureSession的文档描述了哪些保证可以工作。

输出类型

输出类型指的是帧的编码格式。可能的值为 PRIV、YUV、JPEG 和 RAW。 createCaptureSession的文档对它们进行了描述。

在选择应用程序的输出类型时，如果目标是最大限度地提高兼容性，则对于帧分析使用ImageFormat.YUV_420_888，对于静态图像使用ImageFormat.JPEG。对于预览和录制场景，您可能会使用SurfaceView、TextureView、MediaRecorder、MediaCodec或RenderScript.Allocation。在这些情况下，请不要指定图像格式。为了兼容性，它将被视为ImageFormat.PRIVATE，无论内部使用的实际格式是什么。要查询给定其CameraCharacteristics的设备支持的格式，请使用以下代码

Kotlin

val characteristics: CameraCharacteristics = ...
val supportedFormats = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).outputFormats

Java

CameraCharacteristics characteristics = …;
        int[] supportedFormats = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).getOutputFormats();

输出大小

所有可用的输出大小都由StreamConfigurationMap.getOutputSizes()列出，但只有两个与兼容性相关：PREVIEW和MAXIMUM。这些大小充当上限。如果大小为PREVIEW的内容有效，则任何小于PREVIEW大小的内容也将有效。 MAXIMUM也是如此。CameraDevice的文档解释了这些大小。

可用的输出大小取决于格式的选择。给定CameraCharacteristics和格式，您可以像这样查询可用的输出大小

Kotlin

val characteristics: CameraCharacteristics = ...
val outputFormat: Int = ...  // such as ImageFormat.JPEG
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(outputFormat)

Java

CameraCharacteristics characteristics = …;
        int outputFormat = …;  // such as ImageFormat.JPEG
Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

在相机预览和录制用例中，使用目标类确定支持的大小。格式将由相机框架本身处理。

Kotlin

val characteristics: CameraCharacteristics = ...
val targetClass: Class <T> = ...  // such as SurfaceView::class.java
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(targetClass)

Java

CameraCharacteristics characteristics = …;
   int outputFormat = …;  // such as ImageFormat.JPEG
   Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

要获取MAXIMUM大小，请按面积对输出大小进行排序，然后返回最大值。

Kotlin

fun <T>getMaximumOutputSize(
    characteristics: CameraCharacteristics, targetClass: Class <T>, format: Int? = null):
    Size {
  val config = characteristics.get(
      CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)

  // If image format is provided, use it to determine supported sizes; or else use target class
  val allSizes = if (format == null)
    config.getOutputSizes(targetClass) else config.getOutputSizes(format)
  return allSizes.maxBy { it.height * it.width }
}

Java

 @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getMaximumOutputSize(CameraCharacteristics characteristics,
                                            Class <T> targetClass,
                                            Integer format) {
        StreamConfigurationMap config = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);

        // If image format is provided, use it to determine supported sizes; else use target class
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }
        return Arrays.stream(allSizes).max(Comparator.comparing(s -> s.getHeight() * s.getWidth())).get();
    }

PREVIEW指的是与设备屏幕分辨率或 1080p (1920x1080) 最匹配的大小（取较小者）。纵横比可能与屏幕的纵横比不完全匹配，因此您可能需要对流应用信箱或裁剪才能以全屏模式显示它。要获得正确的预览大小，请考虑显示屏可能已旋转的情况，并将可用的输出大小与显示大小进行比较。

以下代码定义了一个辅助类SmartSize，这将使大小比较更容易一些

Kotlin

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize(width: Int, height: Int) {
    var size = Size(width, height)
    var long = max(size.width, size.height)
    var short = min(size.width, size.height)
    override fun toString() = "SmartSize(${long}x${short})"
}

/** Standard High Definition size for pictures and video */
val SIZE_1080P: SmartSize = SmartSize(1920, 1080)

/** Returns a [SmartSize] object for the given [Display] */
fun getDisplaySmartSize(display: Display): SmartSize {
    val outPoint = Point()
    display.getRealSize(outPoint)
    return SmartSize(outPoint.x, outPoint.y)
}

/**
 * Returns the largest available PREVIEW size. For more information, see:
 * https://d.android.com/reference/android/hardware/camera2/CameraDevice
 */
fun <T>getPreviewOutputSize(
        display: Display,
        characteristics: CameraCharacteristics,
        targetClass: Class <T>,
        format: Int? = null
): Size {

    // Find which is smaller: screen or 1080p
    val screenSize = getDisplaySmartSize(display)
    val hdScreen = screenSize.long >= SIZE_1080P.long || screenSize.short >= SIZE_1080P.short
    val maxSize = if (hdScreen) SIZE_1080P else screenSize

    // If image format is provided, use it to determine supported sizes; else use target class
    val config = characteristics.get(
            CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)!!
    if (format == null)
        assert(StreamConfigurationMap.isOutputSupportedFor(targetClass))
    else
        assert(config.isOutputSupportedFor(format))
    val allSizes = if (format == null)
        config.getOutputSizes(targetClass) else config.getOutputSizes(format)

    // Get available sizes and sort them by area from largest to smallest
    val validSizes = allSizes
            .sortedWith(compareBy { it.height * it.width })
            .map { SmartSize(it.width, it.height) }.reversed()

    // Then, get the largest output size that is smaller or equal than our max size
    return validSizes.first { it.long <= maxSize.long && it.short <= maxSize.short }.size
}

Java

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
    class SmartSize {
        Size size;
        double longSize;
        double shortSize;

        public SmartSize(Integer width, Integer height) {
            size = new Size(width, height);
            longSize = max(size.getWidth(), size.getHeight());
            shortSize = min(size.getWidth(), size.getHeight());
        }

        @Override
        public String toString() {
            return String.format("SmartSize(%sx%s)", longSize, shortSize);
        }
    }

    /** Standard High Definition size for pictures and video */
    SmartSize SIZE_1080P = new SmartSize(1920, 1080);

    /** Returns a [SmartSize] object for the given [Display] */
    SmartSize getDisplaySmartSize(Display display) {
        Point outPoint = new Point();
        display.getRealSize(outPoint);
        return new SmartSize(outPoint.x, outPoint.y);
    }

    /**
     * Returns the largest available PREVIEW size. For more information, see:
     * https://d.android.com/reference/android/hardware/camera2/CameraDevice
     */
    @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getPreviewOutputSize(
            Display display,
            CameraCharacteristics characteristics,
            Class <T> targetClass,
            Integer format
    ){

        // Find which is smaller: screen or 1080p
        SmartSize screenSize = getDisplaySmartSize(display);
        boolean hdScreen = screenSize.longSize >= SIZE_1080P.longSize || screenSize.shortSize >= SIZE_1080P.shortSize;
        SmartSize maxSize;
        if (hdScreen) {
            maxSize = SIZE_1080P;
        } else {
            maxSize = screenSize;
        }

        // If image format is provided, use it to determine supported sizes; else use target class
        StreamConfigurationMap config = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
        if (format == null)
            assert(StreamConfigurationMap.isOutputSupportedFor(targetClass));
        else
            assert(config.isOutputSupportedFor(format));
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }

        // Get available sizes and sort them by area from largest to smallest
        List <Size> sortedSizes = Arrays.asList(allSizes);
        List <SmartSize> validSizes =
                sortedSizes.stream()
                        .sorted(Comparator.comparing(s -> s.getHeight() * s.getWidth()))
                        .map(s -> new SmartSize(s.getWidth(), s.getHeight()))
                        .sorted(Collections.reverseOrder()).collect(Collectors.toList());

        // Then, get the largest output size that is smaller or equal than our max size
        return validSizes.stream()
                .filter(s -> s.longSize <= maxSize.longSize && s.shortSize <= maxSize.shortSize)
                .findFirst().get().size;
    }

检查支持的硬件级别

要在运行时确定可用的功能，请使用CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL检查支持的硬件级别。

使用CameraCharacteristics对象，您可以使用单个语句检索硬件级别

Kotlin

val characteristics: CameraCharacteristics = ...

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
val hardwareLevel = characteristics.get(
        CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL)

Java

CameraCharacteristics characteristics = ...;

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
Integer hardwareLevel = characteristics.get(
                CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);

将所有部分放在一起

有了输出类型、输出大小和硬件级别，您可以确定哪些流组合是有效的。下表是具有LEGACY硬件级别的CameraDevice支持的配置的快照。

目标 1		目标 2		目标 3		用例示例
类型	最大大小	类型	最大大小	类型	最大大小	用例示例
`PRIV`	`MAXIMUM`					简单的预览、GPU 视频处理或无预览视频录制。
`JPEG`	`MAXIMUM`					无取景器静态图像捕获。
`YUV`	`MAXIMUM`					应用程序内视频/图像处理。
`PRIV`	`PREVIEW`	`JPEG`	`MAXIMUM`			标准静态图像成像。
`YUV`	`PREVIEW`	`JPEG`	`MAXIMUM`			应用程序内处理加上静态图像捕获。
`PRIV`	`PREVIEW`	`PRIV`	`PREVIEW`			标准录制。
`PRIV`	`PREVIEW`	`YUV`	`PREVIEW`			预览加上应用程序内处理。
`PRIV`	`PREVIEW`	`YUV`	`PREVIEW`			预览加上应用程序内处理。
`PRIV`	`PREVIEW`	`YUV`	`PREVIEW`	`JPEG`	`MAXIMUM`	静态图像捕获加上应用程序内处理。

LEGACY是最低可能的硬件级别。此表显示，每个支持 Camera2（API 级别 21 及更高版本）的设备都可以使用正确的配置输出多达三个同时流，如果不会有太多限制性能的开销（例如内存、CPU 或热限制）。

您的应用还需要配置目标输出缓冲区。例如，要定位具有LEGACY硬件级别的设备，您可以设置两个目标输出表面，一个使用ImageFormat.PRIVATE，另一个使用ImageFormat.YUV_420_888。在使用PREVIEW大小时，这是一个受支持的组合。使用本主题前面定义的函数，获取相机 ID 的所需预览大小需要以下代码

Kotlin

val characteristics: CameraCharacteristics = ...
val context = this as Context  // assuming you are inside of an activity

val surfaceViewSize = getPreviewOutputSize(
    context, characteristics, SurfaceView::class.java)
val imageReaderSize = getPreviewOutputSize(
    context, characteristics, ImageReader::class.java, format = ImageFormat.YUV_420_888)

Java

CameraCharacteristics characteristics = ...;
        Context context = this; // assuming you are inside of an activity

        Size surfaceViewSize = getPreviewOutputSize(
                context, characteristics, SurfaceView.class);
        Size imageReaderSize = getPreviewOutputSize(
                context, characteristics, ImageReader.class, format = ImageFormat.YUV_420_888);

它需要使用提供的回调等待SurfaceView准备好。

Kotlin

val surfaceView = findViewById <SurfaceView>(...)
surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
  override fun surfaceCreated(holder: SurfaceHolder) {
    // You do not need to specify image format, and it will be considered of type PRIV
    // Surface is now ready and you could use it as an output target for CameraSession
  }
  ...
})

Java

SurfaceView surfaceView = findViewById <SurfaceView>(...);

surfaceView.getHolder().addCallback(new SurfaceHolder.Callback() {
            @Override
            public void surfaceCreated(@NonNull SurfaceHolder surfaceHolder) {
                // You do not need to specify image format, and it will be considered of type PRIV
                // Surface is now ready and you could use it as an output target for CameraSession
            }
            ...
        });

您可以通过调用SurfaceHolder.setFixedSize()强制SurfaceView与相机输出大小匹配，或者您可以采用类似于GitHub 上相机示例的“常用模块”中的AutoFitSurfaceView的方法，该方法设置绝对大小，同时考虑纵横比和可用空间，并在触发活动更改时自动调整。

使用具有所需格式的ImageReader设置另一个表面更容易，因为没有回调需要等待。

Kotlin

val frameBufferCount = 3  // just an example, depends on your usage of ImageReader
val imageReader = ImageReader.newInstance(
    imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
    frameBufferCount)

Java

int frameBufferCount = 3;  // just an example, depends on your usage of ImageReader
ImageReader imageReader = ImageReader.newInstance(
                imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
                frameBufferCount);

使用像ImageReader这样的阻塞目标缓冲区时，请在使用后丢弃帧。

Kotlin

imageReader.setOnImageAvailableListener({
  val frame =  it.acquireNextImage()
  // Do something with "frame" here
  it.close()
}, null)

Java

imageReader.setOnImageAvailableListener(listener -> {
            Image frame = listener.acquireNextImage();
            // Do something with "frame" here
            listener.close();
        }, null);

LEGACY硬件级别针对的是最低公分母设备。您可以添加条件分支，并在具有LIMITED硬件级别的设备中为一个输出目标表面使用RECORD大小，甚至可以将其增加到具有FULL硬件级别的设备的MAXIMUM大小。