{"_id":"59bf11b9c3d22f001020a280","project":"578c4badbd223d2000cc1441","version":{"_id":"59bc03d31d2d8d001a34457d","project":"578c4badbd223d2000cc1441","__v":2,"createdAt":"2017-09-15T16:46:11.721Z","releaseDate":"2017-09-15T16:46:11.721Z","categories":["59bc03d31d2d8d001a34457e","59bc03d31d2d8d001a34457f","59bc03d31d2d8d001a344580","59bc03d31d2d8d001a344581","59bc03d31d2d8d001a344582","59bc03d31d2d8d001a344583","59bc284b7c3f420010f965e6"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"2.0.0","version":"2.0.0"},"category":{"_id":"59bc03d31d2d8d001a344581","version":"59bc03d31d2d8d001a34457d","project":"578c4badbd223d2000cc1441","__v":0,"sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-07-18T21:08:45.730Z","from_sync":false,"order":4,"slug":"develop","title":"Develop"},"user":"57bb7e47afc18c0e00529cf3","__v":0,"parentDoc":null,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2017-09-18T00:22:17.631Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":2,"body":"[block:callout]\n{\n  \"type\": \"warning\",\n  \"title\": \"AR only available on iPhone 6s/SE and up with iOS 11+\"\n}\n[/block]\nThe Viro platform supports Augmented Reality (AR) development through various AR-specific components and features. This guide will first give an overview of AR and provide a high-level overview of the components and features that enable developers to build AR experiences.\n[block:api-header]\n{\n  \"title\": \"Developing for AR\"\n}\n[/block]\n## Camera Tracking\nThe Viro platform supports development of 6 degrees-of-freedom (6DOF) AR experiences which means that the platform responds to the user's rotation and position as they move about the world by moving the camera. The platform maintains the same right-handed coordinate system as it does in VR, in that the camera is initially at `[0,0,0]` with a forward vector of `[0, 0, -1]` and up vector of `[0,1,0]`\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/3a0e825-viro_camera_diagram.png\",\n        \"viro_camera_diagram.png\",\n        607,\n        676,\n        \"#3cbbfb\"\n      ]\n    }\n  ]\n}\n[/block]\nWhen the user first enters the AR experience, the camera will remain at `[0,0,0]` while underlying AR system gets its bearing. After a few moments, the camera tracking will complete, the developer should receive a callback on `onTrackingInitialized` in [ViroARScene](doc:viroarscene), and the camera should now track the user's movements around their world.\n\n## Building AR Experiences\n\nThe Viro platform provides a large suite of components that developers can leverage to build their AR experiences. However, unlike traditional 3D rendering and VR, AR experiences are meant to be responsive to the user's real world. In order to accomplish this, the Viro platform provides a few AR specific components and features detailed below:\n\n### AR Components\n\nThese are the components built specifically for and used only in AR on the Viro platform.\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"Component\",\n    \"h-1\": \"Description\",\n    \"0-0\": \"[ViroARSceneNavigator](doc:viroarscenenavigator)\",\n    \"1-0\": \"[ViroARScene](doc:viroarscene)\",\n    \"2-0\": \"[ViroARPlane](doc:viroarplane)\",\n    \"3-0\": \"[ViroARPlaneSelector](doc:viroarplaneselector)\",\n    \"3-1\": \"Convenience component that wraps `ViroARPlane` to allow end users to select the plane they want the developer to use to display content.\",\n    \"2-1\": \"Component that allows and automatically manages the positioning of other Viro components relative to a plane discovered by the AR system.\",\n    \"1-1\": \"Logical container that contains all the components necessary for a singular AR experience. If the desired AR application has multiple \\\"experiences\\\" then multiple `ViroARScene`s may work best.\",\n    \"0-1\": \"Top-Level React Native component that presents a view onto which Viro renders `ViroARScene`s.\"\n  },\n  \"cols\": 2,\n  \"rows\": 4\n}\n[/block]\nAs you can see, the `ViroARSceneNavigator` and `ViroARScene` replace the `ViroSceneNavigator` and `ViroScene` for VR and also add a few more AR-specific methods and properties.\n\nThe `ViroARPlane` and `ViroARPlaneSelector` are the only other two fully-AR components which allow the developer to place their content with respect to planes in the real world.\n\n### AR Features\n\nThe AR system also provides other information which we make available through the following features on our existing components:\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"Features\",\n    \"h-1\": \"Location in API\",\n    \"h-2\": \"Description\",\n    \"0-0\": \"6 Degrees of Camera Movement\",\n    \"0-1\": \"Camera orientation can be accessed through `getCameraOrientationAsync()` in [ViroARScene](doc:viroarscene).\",\n    \"0-2\": \"The camera automatically moves in step with the user's movements in the real world. This results in virtual objects appearing to stay in their positions.\",\n    \"3-0\": \"`FixedToWorld` Dragging\",\n    \"3-1\": \"Supported by all \\\"draggable\\\" components through the `dragType` property.\",\n    \"3-2\": \"Allows the user to drag objects such that they look fixed to points in the real world.\",\n    \"4-0\": \"AR-based Hit Tests\",\n    \"4-1\": \"Methods in [ViroARScene](doc:viroarscene)\",\n    \"4-2\": \"Allows to user to get points in the real world from the AR system.\",\n    \"2-0\": \"Ambient Light Estimation\",\n    \"2-1\": \"`onAmbientLightUpdate` in [ViroARScene](doc:viroarscene)\",\n    \"2-2\": \"Provides an estimate of light intensity and color temperature as detected by the camera.\",\n    \"1-0\": \"Video Pass Through (rendered as background)\",\n    \"1-1\": \"Automatic\",\n    \"1-2\": \"In AR, the back camera is enabled and serves as the \\\"background\\\" of the view onto which virtual objects are rendered.\",\n    \"5-0\": \"Portals\",\n    \"5-1\": \"[ViroPortalScene](doc:viroportalscene) and [ViroPortal](doc:viroportal) components\",\n    \"5-2\": \"Allows the developer to add a virtual \\\"portal\\\" to the user's real world.\",\n    \"6-0\": \"Post processing effects\",\n    \"6-1\": \"`postProcessEffects` prop in [ViroARScene](doc:viroarscene)\",\n    \"6-2\": \"Allows the developer to apply a pre-built post process effect on their scene.\",\n    \"7-0\": \"Video and still capture\",\n    \"7-1\": \"Recording API's in [ViroARSceneNavigator](doc:viroarscenenavigator)\",\n    \"7-2\": \"Allows the developer to easily record their virtual objects on top of the recorded camera view.\"\n  },\n  \"cols\": 3,\n  \"rows\": 8\n}\n[/block]\nWith the above features, developers can illuminate their AR experiences with more realistic lighting, use realistic object movement, add other-worldly portals and much more.","excerpt":"","slug":"augmented-reality-ar","type":"basic","title":"Augmented Reality (AR)"}

Augmented Reality (AR)


[block:callout] { "type": "warning", "title": "AR only available on iPhone 6s/SE and up with iOS 11+" } [/block] The Viro platform supports Augmented Reality (AR) development through various AR-specific components and features. This guide will first give an overview of AR and provide a high-level overview of the components and features that enable developers to build AR experiences. [block:api-header] { "title": "Developing for AR" } [/block] ## Camera Tracking The Viro platform supports development of 6 degrees-of-freedom (6DOF) AR experiences which means that the platform responds to the user's rotation and position as they move about the world by moving the camera. The platform maintains the same right-handed coordinate system as it does in VR, in that the camera is initially at `[0,0,0]` with a forward vector of `[0, 0, -1]` and up vector of `[0,1,0]` [block:image] { "images": [ { "image": [ "https://files.readme.io/3a0e825-viro_camera_diagram.png", "viro_camera_diagram.png", 607, 676, "#3cbbfb" ] } ] } [/block] When the user first enters the AR experience, the camera will remain at `[0,0,0]` while underlying AR system gets its bearing. After a few moments, the camera tracking will complete, the developer should receive a callback on `onTrackingInitialized` in [ViroARScene](doc:viroarscene), and the camera should now track the user's movements around their world. ## Building AR Experiences The Viro platform provides a large suite of components that developers can leverage to build their AR experiences. However, unlike traditional 3D rendering and VR, AR experiences are meant to be responsive to the user's real world. In order to accomplish this, the Viro platform provides a few AR specific components and features detailed below: ### AR Components These are the components built specifically for and used only in AR on the Viro platform. [block:parameters] { "data": { "h-0": "Component", "h-1": "Description", "0-0": "[ViroARSceneNavigator](doc:viroarscenenavigator)", "1-0": "[ViroARScene](doc:viroarscene)", "2-0": "[ViroARPlane](doc:viroarplane)", "3-0": "[ViroARPlaneSelector](doc:viroarplaneselector)", "3-1": "Convenience component that wraps `ViroARPlane` to allow end users to select the plane they want the developer to use to display content.", "2-1": "Component that allows and automatically manages the positioning of other Viro components relative to a plane discovered by the AR system.", "1-1": "Logical container that contains all the components necessary for a singular AR experience. If the desired AR application has multiple \"experiences\" then multiple `ViroARScene`s may work best.", "0-1": "Top-Level React Native component that presents a view onto which Viro renders `ViroARScene`s." }, "cols": 2, "rows": 4 } [/block] As you can see, the `ViroARSceneNavigator` and `ViroARScene` replace the `ViroSceneNavigator` and `ViroScene` for VR and also add a few more AR-specific methods and properties. The `ViroARPlane` and `ViroARPlaneSelector` are the only other two fully-AR components which allow the developer to place their content with respect to planes in the real world. ### AR Features The AR system also provides other information which we make available through the following features on our existing components: [block:parameters] { "data": { "h-0": "Features", "h-1": "Location in API", "h-2": "Description", "0-0": "6 Degrees of Camera Movement", "0-1": "Camera orientation can be accessed through `getCameraOrientationAsync()` in [ViroARScene](doc:viroarscene).", "0-2": "The camera automatically moves in step with the user's movements in the real world. This results in virtual objects appearing to stay in their positions.", "3-0": "`FixedToWorld` Dragging", "3-1": "Supported by all \"draggable\" components through the `dragType` property.", "3-2": "Allows the user to drag objects such that they look fixed to points in the real world.", "4-0": "AR-based Hit Tests", "4-1": "Methods in [ViroARScene](doc:viroarscene)", "4-2": "Allows to user to get points in the real world from the AR system.", "2-0": "Ambient Light Estimation", "2-1": "`onAmbientLightUpdate` in [ViroARScene](doc:viroarscene)", "2-2": "Provides an estimate of light intensity and color temperature as detected by the camera.", "1-0": "Video Pass Through (rendered as background)", "1-1": "Automatic", "1-2": "In AR, the back camera is enabled and serves as the \"background\" of the view onto which virtual objects are rendered.", "5-0": "Portals", "5-1": "[ViroPortalScene](doc:viroportalscene) and [ViroPortal](doc:viroportal) components", "5-2": "Allows the developer to add a virtual \"portal\" to the user's real world.", "6-0": "Post processing effects", "6-1": "`postProcessEffects` prop in [ViroARScene](doc:viroarscene)", "6-2": "Allows the developer to apply a pre-built post process effect on their scene.", "7-0": "Video and still capture", "7-1": "Recording API's in [ViroARSceneNavigator](doc:viroarscenenavigator)", "7-2": "Allows the developer to easily record their virtual objects on top of the recorded camera view." }, "cols": 3, "rows": 8 } [/block] With the above features, developers can illuminate their AR experiences with more realistic lighting, use realistic object movement, add other-worldly portals and much more.