注意:此页面指的是 Camera2 软件包。除非您的应用需要 Camera2 的特定低级功能,否则我们建议使用 CameraX。CameraX 和 Camera2 都支持 Android 5.0(API 级别 21)及更高版本。
多摄像头功能是在 Android 9(API 级别 28)中引入的。自发布以来,市场上已经出现了支持该 API 的设备。许多多摄像头用例与特定的硬件配置紧密相关。换句话说,并非所有用例都与每台设备兼容,这使得多摄像头功能成为 Play 功能交付 的良好选择。
一些典型的用例包括
- 缩放:根据裁剪区域或所需的焦距在摄像头之间切换。
- 深度:使用多个摄像头构建深度图。
- 散景:使用推断的深度信息来模拟类似单反相机的窄焦距范围。
逻辑摄像头和物理摄像头的区别
理解多摄像头 API 需要了解逻辑摄像头和物理摄像头的区别。作为参考,考虑一个具有三个后置摄像头的设备。在这个示例中,三个后置摄像头中的每一个都被视为一个物理摄像头。然后,逻辑摄像头是这些物理摄像头中的两个或多个的组合。逻辑摄像头的输出可以是来自其中一个底层物理摄像头的流,也可以是同时来自多个底层物理摄像头的融合流。无论哪种方式,该流都由摄像头硬件抽象层 (HAL) 处理。
许多手机制造商开发第一方摄像头应用程序,这些应用程序通常预装在他们的设备上。为了使用所有硬件的功能,他们可能会使用私有或隐藏的 API,或者从驱动程序实现获得其他应用程序无法访问的特殊待遇。某些设备通过提供来自不同物理摄像头的帧的融合流来实现逻辑摄像头的概念,但仅限于某些特权应用程序。通常,只有其中一个物理摄像头暴露给框架。Android 9 之前的第三方开发人员的情况如下所示
从 Android 9 开始,Android 应用程序不再允许使用私有 API。随着框架中包含了多摄像头支持,Android 最佳实践强烈建议手机制造商为所有朝向相同方向的物理摄像头公开一个逻辑摄像头。以下是第三方开发人员在运行 Android 9 及更高版本的设备上应该看到的内容
逻辑摄像头提供的内容完全取决于摄像头 HAL 的 OEM 实现。例如,像 Pixel 3 这样的设备以这样一种方式实现其逻辑摄像头:它根据请求的焦距和裁剪区域选择其物理摄像头之一。
多摄像头 API
新的 API 添加了以下新的常量、类和方法
CameraMetadata.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA
CameraCharacteristics.getPhysicalCameraIds()
CameraCharacteristics.getAvailablePhysicalCameraRequestKeys()
CameraDevice.createCaptureSession(SessionConfiguration config)
CameraCharacteritics.LOGICAL_MULTI_CAMERA_SENSOR_SYNC_TYPE
OutputConfiguration
和SessionConfiguration
由于 Android 兼容性定义文档 (CDD) 的更改,多摄像头 API 还附带了开发人员的一些期望。在 Android 9 之前就存在双摄像头的设备,但同时打开多个摄像头涉及反复试验。在 Android 9 及更高版本上,多摄像头提供了一组规则来指定何时可以打开属于同一逻辑摄像头的两个物理摄像头。
在大多数情况下,运行 Android 9 及更高版本的设备会公开所有物理摄像头(可能除了不太常见的传感器类型,如红外线)以及一个更易于使用的逻辑摄像头。对于保证可以工作的每种流组合,都可以用来自底层物理摄像头的两个流替换属于逻辑摄像头的其中一个流。
同时使用多个流
同时使用多个摄像头流 涵盖了在单个摄像头中同时使用多个流的规则。除了一个显著的补充之外,相同的规则适用于多个摄像头。 CameraMetadata.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA
解释了如何用两个物理流替换逻辑 YUV_420_888 或原始流。也就是说,每种类型为 YUV 或 RAW 的流都可以替换为两个类型和大小相同的流。您可以从以下单摄像头设备的保证配置的摄像头流开始
- 流 1:YUV 类型,
MAXIMUM
大小来自逻辑摄像头id = 0
然后,支持多摄像头的设备允许您创建替换该逻辑 YUV 流的会话,使用两个物理流
- 流 1:YUV 类型,
MAXIMUM
大小来自物理摄像头id = 1
- 流 2:YUV 类型,
MAXIMUM
大小来自物理摄像头id = 2
当且仅当这两个摄像头是逻辑摄像头分组的一部分时,才能用两个等效的流替换 YUV 或 RAW 流,这在 CameraCharacteristics.getPhysicalCameraIds()
中列出。
框架提供的保证只是同时从多个物理摄像头获取帧所需的最低限度。大多数设备支持其他流,有时甚至允许独立打开多个物理摄像头设备。由于这不是框架的硬性保证,因此这样做需要使用反复试验进行逐设备测试和调整。
使用多个物理摄像头创建会话
在启用多摄像头的设备上使用物理摄像头时,打开单个 CameraDevice
(逻辑摄像头)并在单个会话中与其交互。使用 API CameraDevice.createCaptureSession(SessionConfiguration config)
创建单个会话,该 API 在 API 级别 28 中添加。会话配置具有多个输出配置,每个配置都有一组输出目标,并且可以选择一个所需的物理摄像头 ID。
捕获请求与其关联的输出目标相关联。框架根据附加的输出目标确定将请求发送到哪个物理(或逻辑)摄像头。如果输出目标对应于作为输出配置以及物理摄像头 ID 发送的输出目标之一,则该物理摄像头将接收并处理该请求。
使用一对物理摄像头
多摄像头的摄像头 API 的另一个补充功能是能够识别逻辑摄像头并找到其背后的物理摄像头。您可以定义一个函数来帮助识别可以用来替换逻辑摄像头流之一的潜在物理摄像头对
Kotlin
/** * Helper class used to encapsulate a logical camera and two underlying * physical cameras */ data class DualCamera(val logicalId: String, val physicalId1: String, val physicalId2: String) fun findDualCameras(manager: CameraManager, facing: Int? = null): List{ val dualCameras = MutableList () // Iterate over all the available camera characteristics manager.cameraIdList.map { Pair(manager.getCameraCharacteristics(it), it) }.filter { // Filter by cameras facing the requested direction facing == null || it.first.get(CameraCharacteristics.LENS_FACING) == facing }.filter { // Filter by logical cameras // CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA requires API >= 28 it.first.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES)!!.contains( CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA) }.forEach { // All possible pairs from the list of physical cameras are valid results // NOTE: There could be N physical cameras as part of a logical camera grouping // getPhysicalCameraIds() requires API >= 28 val physicalCameras = it.first.physicalCameraIds.toTypedArray() for (idx1 in 0 until physicalCameras.size) { for (idx2 in (idx1 + 1) until physicalCameras.size) { dualCameras.add(DualCamera( it.second, physicalCameras[idx1], physicalCameras[idx2])) } } } return dualCameras }
Java
/** * Helper class used to encapsulate a logical camera and two underlying * physical cameras */ final class DualCamera { final String logicalId; final String physicalId1; final String physicalId2; DualCamera(String logicalId, String physicalId1, String physicalId2) { this.logicalId = logicalId; this.physicalId1 = physicalId1; this.physicalId2 = physicalId2; } } ListfindDualCameras(CameraManager manager, Integer facing) { List dualCameras = new ArrayList<>(); List cameraIdList; try { cameraIdList = Arrays.asList(manager.getCameraIdList()); } catch (CameraAccessException e) { e.printStackTrace(); cameraIdList = new ArrayList<>(); } // Iterate over all the available camera characteristics cameraIdList.stream() .map(id -> { try { CameraCharacteristics characteristics = manager.getCameraCharacteristics(id); return new Pair<>(characteristics, id); } catch (CameraAccessException e) { e.printStackTrace(); return null; } }) .filter(pair -> { // Filter by cameras facing the requested direction return (pair != null) && (facing == null || pair.first.get(CameraCharacteristics.LENS_FACING).equals(facing)); }) .filter(pair -> { // Filter by logical cameras // CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA requires API >= 28 IntPredicate logicalMultiCameraPred = arg -> arg == CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA; return Arrays.stream(pair.first.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES)) .anyMatch(logicalMultiCameraPred); }) .forEach(pair -> { // All possible pairs from the list of physical cameras are valid results // NOTE: There could be N physical cameras as part of a logical camera grouping // getPhysicalCameraIds() requires API >= 28 String[] physicalCameras = pair.first.getPhysicalCameraIds().toArray(new String[0]); for (int idx1 = 0; idx1 < physicalCameras.length; idx1++) { for (int idx2 = idx1 + 1; idx2 < physicalCameras.length; idx2++) { dualCameras.add( new DualCamera(pair.second, physicalCameras[idx1], physicalCameras[idx2])); } } }); return dualCameras; }
物理摄像头的状态处理由逻辑摄像头控制。要打开“双摄像头”,请打开与物理摄像头相对应的逻辑摄像头
Kotlin
fun openDualCamera(cameraManager: CameraManager, dualCamera: DualCamera, // AsyncTask is deprecated beginning API 30 executor: Executor = AsyncTask.SERIAL_EXECUTOR, callback: (CameraDevice) -> Unit) { // openCamera() requires API >= 28 cameraManager.openCamera( dualCamera.logicalId, executor, object : CameraDevice.StateCallback() { override fun onOpened(device: CameraDevice) = callback(device) // Omitting for brevity... override fun onError(device: CameraDevice, error: Int) = onDisconnected(device) override fun onDisconnected(device: CameraDevice) = device.close() }) }
Java
void openDualCamera(CameraManager cameraManager, DualCamera dualCamera, Executor executor, CameraDeviceCallback cameraDeviceCallback ) { // openCamera() requires API >= 28 cameraManager.openCamera(dualCamera.logicalId, executor, new CameraDevice.StateCallback() { @Override public void onOpened(@NonNull CameraDevice cameraDevice) { cameraDeviceCallback.callback(cameraDevice); } @Override public void onDisconnected(@NonNull CameraDevice cameraDevice) { cameraDevice.close(); } @Override public void onError(@NonNull CameraDevice cameraDevice, int i) { onDisconnected(cameraDevice); } }); }
除了选择要打开的摄像头之外,该过程与在过去的 Android 版本中打开摄像头相同。使用新的会话配置 API 创建捕获会话会告诉框架将某些目标与特定的物理摄像头 ID 关联起来
Kotlin
/** * Helper type definition that encapsulates 3 sets of output targets: * * 1. Logical camera * 2. First physical camera * 3. Second physical camera */ typealias DualCameraOutputs = Triple?, MutableList ?, MutableList ?> fun createDualCameraSession(cameraManager: CameraManager, dualCamera: DualCamera, targets: DualCameraOutputs, // AsyncTask is deprecated beginning API 30 executor: Executor = AsyncTask.SERIAL_EXECUTOR, callback: (CameraCaptureSession) -> Unit) { // Create 3 sets of output configurations: one for the logical camera, and // one for each of the physical cameras. val outputConfigsLogical = targets.first?.map { OutputConfiguration(it) } val outputConfigsPhysical1 = targets.second?.map { OutputConfiguration(it).apply { setPhysicalCameraId(dualCamera.physicalId1) } } val outputConfigsPhysical2 = targets.third?.map { OutputConfiguration(it).apply { setPhysicalCameraId(dualCamera.physicalId2) } } // Put all the output configurations into a single flat array val outputConfigsAll = arrayOf( outputConfigsLogical, outputConfigsPhysical1, outputConfigsPhysical2) .filterNotNull().flatMap { it } // Instantiate a session configuration that can be used to create a session val sessionConfiguration = SessionConfiguration( SessionConfiguration.SESSION_REGULAR, outputConfigsAll, executor, object : CameraCaptureSession.StateCallback() { override fun onConfigured(session: CameraCaptureSession) = callback(session) // Omitting for brevity... override fun onConfigureFailed(session: CameraCaptureSession) = session.device.close() }) // Open the logical camera using the previously defined function openDualCamera(cameraManager, dualCamera, executor = executor) { // Finally create the session and return via callback it.createCaptureSession(sessionConfiguration) } }
Java
/** * Helper class definition that encapsulates 3 sets of output targets: ** 1. Logical camera * 2. First physical camera * 3. Second physical camera */ final class DualCameraOutputs { private final List
logicalCamera; private final List firstPhysicalCamera; private final List secondPhysicalCamera; public DualCameraOutputs(List logicalCamera, List firstPhysicalCamera, List third) { this.logicalCamera = logicalCamera; this.firstPhysicalCamera = firstPhysicalCamera; this.secondPhysicalCamera = third; } public List getLogicalCamera() { return logicalCamera; } public List getFirstPhysicalCamera() { return firstPhysicalCamera; } public List getSecondPhysicalCamera() { return secondPhysicalCamera; } } interface CameraCaptureSessionCallback { void callback(CameraCaptureSession cameraCaptureSession); } void createDualCameraSession(CameraManager cameraManager, DualCamera dualCamera, DualCameraOutputs targets, Executor executor, CameraCaptureSessionCallback cameraCaptureSessionCallback) { // Create 3 sets of output configurations: one for the logical camera, and // one for each of the physical cameras. List outputConfigsLogical = targets.getLogicalCamera().stream() .map(OutputConfiguration::new) .collect(Collectors.toList()); List outputConfigsPhysical1 = targets.getFirstPhysicalCamera().stream() .map(s -> { OutputConfiguration outputConfiguration = new OutputConfiguration(s); outputConfiguration.setPhysicalCameraId(dualCamera.physicalId1); return outputConfiguration; }) .collect(Collectors.toList()); List outputConfigsPhysical2 = targets.getSecondPhysicalCamera().stream() .map(s -> { OutputConfiguration outputConfiguration = new OutputConfiguration(s); outputConfiguration.setPhysicalCameraId(dualCamera.physicalId2); return outputConfiguration; }) .collect(Collectors.toList()); // Put all the output configurations into a single flat array List outputConfigsAll = Stream.of( outputConfigsLogical, outputConfigsPhysical1, outputConfigsPhysical2 ) .filter(Objects::nonNull) .flatMap(Collection::stream) .collect(Collectors.toList()); // Instantiate a session configuration that can be used to create a session SessionConfiguration sessionConfiguration = new SessionConfiguration( SessionConfiguration.SESSION_REGULAR, outputConfigsAll, executor, new CameraCaptureSession.StateCallback() { @Override public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) { cameraCaptureSessionCallback.callback(cameraCaptureSession); } // Omitting for brevity... @Override public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) { cameraCaptureSession.getDevice().close(); } }); // Open the logical camera using the previously defined function openDualCamera(cameraManager, dualCamera, executor, (CameraDevice c) -> // Finally create the session and return via callback c.createCaptureSession(sessionConfiguration)); }
有关支持的流组合的信息,请参阅 createCaptureSession
。流组合用于单个逻辑摄像头上的多个流。兼容性扩展到使用相同的配置并将其中一个流替换为来自属于同一逻辑摄像头的两个物理摄像头的两个流。
准备就绪后,使用 摄像头会话 分派所需的 捕获请求。如果任何物理摄像头正在使用,则捕获请求的每个目标都会从其关联的物理摄像头接收数据,否则将回退到逻辑摄像头。
缩放用例示例
可以将物理摄像头合并到单个流中,以便用户可以在不同的物理摄像头之间切换以体验不同的视野,从而有效地捕获不同的“缩放级别”。
首先选择要允许用户在其之间切换的一对物理摄像头。为了获得最大效果,您可以选择提供可用最小和最大焦距的一对摄像头。
Kotlin
fun findShortLongCameraPair(manager: CameraManager, facing: Int? = null): DualCamera? { return findDualCameras(manager, facing).map { val characteristics1 = manager.getCameraCharacteristics(it.physicalId1) val characteristics2 = manager.getCameraCharacteristics(it.physicalId2) // Query the focal lengths advertised by each physical camera val focalLengths1 = characteristics1.get( CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS) ?: floatArrayOf(0F) val focalLengths2 = characteristics2.get( CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS) ?: floatArrayOf(0F) // Compute the largest difference between min and max focal lengths between cameras val focalLengthsDiff1 = focalLengths2.maxOrNull()!! - focalLengths1.minOrNull()!! val focalLengthsDiff2 = focalLengths1.maxOrNull()!! - focalLengths2.minOrNull()!! // Return the pair of camera IDs and the difference between min and max focal lengths if (focalLengthsDiff1 < focalLengthsDiff2) { Pair(DualCamera(it.logicalId, it.physicalId1, it.physicalId2), focalLengthsDiff1) } else { Pair(DualCamera(it.logicalId, it.physicalId2, it.physicalId1), focalLengthsDiff2) } // Return only the pair with the largest difference, or null if no pairs are found }.maxByOrNull { it.second }?.first }
Java
// Utility functions to find min/max value in float[] float findMax(float[] array) { float max = Float.NEGATIVE_INFINITY; for(float cur: array) max = Math.max(max, cur); return max; } float findMin(float[] array) { float min = Float.NEGATIVE_INFINITY; for(float cur: array) min = Math.min(min, cur); return min; } DualCamera findShortLongCameraPair(CameraManager manager, Integer facing) { return findDualCameras(manager, facing).stream() .map(c -> { CameraCharacteristics characteristics1; CameraCharacteristics characteristics2; try { characteristics1 = manager.getCameraCharacteristics(c.physicalId1); characteristics2 = manager.getCameraCharacteristics(c.physicalId2); } catch (CameraAccessException e) { e.printStackTrace(); return null; } // Query the focal lengths advertised by each physical camera float[] focalLengths1 = characteristics1.get( CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS); float[] focalLengths2 = characteristics2.get( CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS); // Compute the largest difference between min and max focal lengths between cameras Float focalLengthsDiff1 = findMax(focalLengths2) - findMin(focalLengths1); Float focalLengthsDiff2 = findMax(focalLengths1) - findMin(focalLengths2); // Return the pair of camera IDs and the difference between min and max focal lengths if (focalLengthsDiff1 < focalLengthsDiff2) { return new Pair<>(new DualCamera(c.logicalId, c.physicalId1, c.physicalId2), focalLengthsDiff1); } else { return new Pair<>(new DualCamera(c.logicalId, c.physicalId2, c.physicalId1), focalLengthsDiff2); } }) // Return only the pair with the largest difference, or null if no pairs are found .max(Comparator.comparing(pair -> pair.second)).get().first; }
对此的合理架构是拥有两个 SurfaceViews
,每个流一个。这些 SurfaceViews
根据用户交互进行交换,以便在任何给定时间只有一个可见。
以下代码显示了如何打开逻辑摄像头、配置摄像头输出、创建摄像头会话以及启动两个预览流
Kotlin
val cameraManager: CameraManager = ... // Get the two output targets from the activity / fragment val surface1 = ... // from SurfaceView val surface2 = ... // from SurfaceView val dualCamera = findShortLongCameraPair(manager)!! val outputTargets = DualCameraOutputs( null, mutableListOf(surface1), mutableListOf(surface2)) // Here you open the logical camera, configure the outputs and create a session createDualCameraSession(manager, dualCamera, targets = outputTargets) { session -> // Create a single request which has one target for each physical camera // NOTE: Each target receive frames from only its associated physical camera val requestTemplate = CameraDevice.TEMPLATE_PREVIEW val captureRequest = session.device.createCaptureRequest(requestTemplate).apply { arrayOf(surface1, surface2).forEach { addTarget(it) } }.build() // Set the sticky request for the session and you are done session.setRepeatingRequest(captureRequest, null, null) }
Java
CameraManager manager = ...; // Get the two output targets from the activity / fragment Surface surface1 = ...; // from SurfaceView Surface surface2 = ...; // from SurfaceView DualCamera dualCamera = findShortLongCameraPair(manager, null); DualCameraOutputs outputTargets = new DualCameraOutputs( null, Collections.singletonList(surface1), Collections.singletonList(surface2)); // Here you open the logical camera, configure the outputs and create a session createDualCameraSession(manager, dualCamera, outputTargets, null, (session) -> { // Create a single request which has one target for each physical camera // NOTE: Each target receive frames from only its associated physical camera CaptureRequest.Builder captureRequestBuilder; try { captureRequestBuilder = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW); Arrays.asList(surface1, surface2).forEach(captureRequestBuilder::addTarget); // Set the sticky request for the session and you are done session.setRepeatingRequest(captureRequestBuilder.build(), null, null); } catch (CameraAccessException e) { e.printStackTrace(); } });
剩下的就是为用户提供一个在两个界面之间切换的 UI,例如按钮或双击 SurfaceView
。您甚至可以执行某种形式的场景分析并自动在两个流之间切换。
镜头畸变
所有镜头都会产生一定程度的畸变。在 Android 中,您可以使用 CameraCharacteristics.LENS_DISTORTION
查询镜头产生的畸变,它取代了现已弃用的 CameraCharacteristics.LENS_RADIAL_DISTORTION
。对于逻辑摄像头,畸变最小,您的应用程序或多或少可以使用来自摄像头的帧。对于物理摄像头,可能存在非常不同的镜头配置,尤其是在广角镜头上。
某些设备可以通过 CaptureRequest.DISTORTION_CORRECTION_MODE
实现自动畸变校正。对于大多数设备,畸变校正默认为开启状态。
Kotlin
val cameraSession: CameraCaptureSession = ... // Use still capture template to build the capture request val captureRequest = cameraSession.device.createCaptureRequest( CameraDevice.TEMPLATE_STILL_CAPTURE ) // Determine if this device supports distortion correction val characteristics: CameraCharacteristics = ... val supportsDistortionCorrection = characteristics.get( CameraCharacteristics.DISTORTION_CORRECTION_AVAILABLE_MODES )?.contains( CameraMetadata.DISTORTION_CORRECTION_MODE_HIGH_QUALITY ) ?: false if (supportsDistortionCorrection) { captureRequest.set( CaptureRequest.DISTORTION_CORRECTION_MODE, CameraMetadata.DISTORTION_CORRECTION_MODE_HIGH_QUALITY ) } // Add output target, set other capture request parameters... // Dispatch the capture request cameraSession.capture(captureRequest.build(), ...)
Java
CameraCaptureSession cameraSession = ...; // Use still capture template to build the capture request CaptureRequest.Builder captureRequestBuilder = null; try { captureRequestBuilder = cameraSession.getDevice().createCaptureRequest( CameraDevice.TEMPLATE_STILL_CAPTURE ); } catch (CameraAccessException e) { e.printStackTrace(); } // Determine if this device supports distortion correction CameraCharacteristics characteristics = ...; boolean supportsDistortionCorrection = Arrays.stream( characteristics.get( CameraCharacteristics.DISTORTION_CORRECTION_AVAILABLE_MODES )) .anyMatch(i -> i == CameraMetadata.DISTORTION_CORRECTION_MODE_HIGH_QUALITY); if (supportsDistortionCorrection) { captureRequestBuilder.set( CaptureRequest.DISTORTION_CORRECTION_MODE, CameraMetadata.DISTORTION_CORRECTION_MODE_HIGH_QUALITY ); } // Add output target, set other capture request parameters... // Dispatch the capture request cameraSession.capture(captureRequestBuilder.build(), ...);
以这种模式设置捕获请求可能会影响摄像头可以产生的帧率。您可以选择仅在静态图像捕获时打开畸变校正。