{"_id":"5bec8a15287d1b00426efe49","project":"578c4badbd223d2000cc1441","version":{"_id":"5bec8a15287d1b00426efe54","project":"578c4badbd223d2000cc1441","__v":0,"forked_from":"5ba2b5747091250003437507","createdAt":"2018-04-23T20:03:35.726Z","releaseDate":"2018-04-23T20:03:35.726Z","categories":["5bec8a15287d1b00426efdc6","5bec8a15287d1b00426efdc7","5bec8a15287d1b00426efdc8","5bec8a15287d1b00426efdc9","5bec8a15287d1b00426efdca","5bec8a15287d1b00426efdcb","5bec8a15287d1b00426efdcc","5bec8a15287d1b00426efdcd","5bec8a15287d1b00426efdce"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"2.12.0","version":"2.12.0"},"category":{"_id":"5bec8a15287d1b00426efdcd","project":"578c4badbd223d2000cc1441","version":"5bec8a15287d1b00426efe54","__v":0,"sync":{"url":"","isSync":false},"reference":false,"createdAt":"2018-05-24T22:57:04.021Z","from_sync":false,"order":4,"slug":"augmented-reality","title":"Augmented Reality"},"user":"579a69d53de0a217007eda56","githubsync":"","__v":0,"parentDoc":null,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2018-05-24T23:02:15.079Z","link_external":false,"link_url":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":4,"body":"Image recognition is a key component of AR: it enables you to *interpret* the real world and *respond* to it accordingly. This guide provides an overview of Viro's image recognition capabilities.\n[block:api-header]\n{\n  \"title\": \"Image Targets\"\n}\n[/block]\nImage targets are the reference images that Viro will recognize and track. For example, if you provide an image target of a Tesla logo, then every time your application encounters a Tesla logo you can get a callback; and in that callback, you can create virtual UI about the logo. An example of this is below:\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/3269a8e-viro_car_marker_demo.gif\",\n        \"viro_car_marker_demo.gif\",\n        600,\n        338,\n        \"#9c826b\"\n      ]\n    }\n  ]\n}\n[/block]\nImage targets in Viro are represented by [ViroARTrackingTargets](doc:viroartrackingtargets). You can construct these targets out of any image (JPG, PNG, etc.). To start searching for the logos, add a [<ViroARImageMarker>](doc:viroarimagemarker) component to your ```<ViroARScene>``` or to any ```<ViroNode>```.  When Viro detects the image in the user's world, the content of the ```<ViroARImageMarker>``` will be rendered. The ```<ViroARImageMarker>``` will be continually tracked and will stay synchronized with the detected image. Below is an example of simple usage:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"// In your render function, add an image marker that references the target\\n<ViroARScene>\\n  <ViroARImageMarker target={\\\"targetOne\\\"} >\\n    <ViroBox position={[0, .25, 0]} scale={[.5, .5, .5]} />\\n  </ViroARImageMarker>\\n</ViroARScene>\\n\\n// Outside of the render function, register the target\\n\\nViroARTrackingTargets.createTargets({\\n  \\\"targetOne\\\" : {\\n    source : require('./res/targetOne.jpg'),\\n    orientation : \\\"Up\\\",\\n    physicalWidth : 0.1 // real world width in meters\\n  },\\n});\\n\\n\",\n      \"language\": \"javascript\"\n    }\n  ]\n}\n[/block]\nIn the more complex example below, we do this with a \"Black Panther\" movie poster. Upon detecting the poster, we load a 3D object representing the Black Panther, and make him jump out of the poster.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"// In your render function:\\n\\n<ViroARScene>\\n   <ViroAmbientLight color=\\\"#ffffff\\\" intensity={200}/>\\n   <ViroARImageMarker target={\\\"poster\\\"} \\n                      onAnchorFound={this._onAnchorFound}\\n                      pauseUpdates={this.state.pauseUpdates}>\\n\\n   <ViroNode position={[0, -.1, 0]} scale={[0,0,0]} rotation={[-90, 0, 0]} \\n             dragType=\\\"FixedToWorld\\\" onDrag={()=>{}}\\n             animation={{name:\\\"scaleModel\\\", run:this.state.playAnim}}>\\n                 \\n       <Viro3DObject onLoadEnd={this._onModelLoad}\\n                     source={require('./res/blackpanther/object_bpanther_anim.vrx')}\\n                     position={[0, -1.45, 0]}\\n                     scale={[.9,.9,.9]}\\n                     animation={{name:\\\"01\\\", run:true, loop:false,                                                 onFinish:this._onFinish}}\\n                     type=\\\"VRX\\\" />\\n     </ViroNode>\\n</ViroARImageMarker>\\n   \\n// Outside the render function:               \\n    \\nViroARTrackingTargets.createTargets({\\n  poster : {\\n    source : require('./res/blackpanther.jpg'),\\n    orientation : \\\"Up\\\",\\n    physicalWidth : 0.6096 // real world width in meters\\n  }\\n});\",\n      \"language\": \"javascript\"\n    }\n  ]\n}\n[/block]\nThe full code for this sample is contained in the [Viro Samples](https://github.com/viromedia/viro/tree/master/js/ARPosterDemo) repository. The final result will look like this:\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/f56f163-viro_black_panther_marker_demo.gif\",\n        \"viro_black_panther_marker_demo.gif\",\n        600,\n        278,\n        \"#4b4547\"\n      ]\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"title\": \"Continuous Image Tracking (iOS 12 Only)\"\n}\n[/block]\n\nARKit 2.0 introduced a new API that enables continuous Image Tracking instead of simply image detection. This enables your marker to be tracked and followed smoothly as the user moves it around. \n\nThis is exposed in [ViroARSceneNavigator's](doc:viroarscenenavigator) new **numberOfTrackedImages** property which takes in a number of images that should be tracked concurrently. For example, if this number is set to 3, then the first 3 images visible in a scene will be tracked. Even if there are 5 total ViroARImageMarkers only the first 3 will be tracked, if one marker leaves the view, then an untracked marker will then be tracked. Keep in mind that this number should be kept low as the higher the number, the worse the performance.\n[block:api-header]\n{\n  \"title\": \"Image Target Quality\"\n}\n[/block]\nGoogle has released a tool (for OSX and Windows) to check the quality of Image Targets, check it out on their website here:\n\nhttps://developers.google.com/ar/develop/c/augmented-images/arcoreimg\n\nThis tool provides a good baseline for Image Recognition on both iOS and Android","excerpt":"","slug":"ar-image-recognition","type":"basic","title":"Image Recognition"}
Image recognition is a key component of AR: it enables you to *interpret* the real world and *respond* to it accordingly. This guide provides an overview of Viro's image recognition capabilities. [block:api-header] { "title": "Image Targets" } [/block] Image targets are the reference images that Viro will recognize and track. For example, if you provide an image target of a Tesla logo, then every time your application encounters a Tesla logo you can get a callback; and in that callback, you can create virtual UI about the logo. An example of this is below: [block:image] { "images": [ { "image": [ "https://files.readme.io/3269a8e-viro_car_marker_demo.gif", "viro_car_marker_demo.gif", 600, 338, "#9c826b" ] } ] } [/block] Image targets in Viro are represented by [ViroARTrackingTargets](doc:viroartrackingtargets). You can construct these targets out of any image (JPG, PNG, etc.). To start searching for the logos, add a [<ViroARImageMarker>](doc:viroarimagemarker) component to your ```<ViroARScene>``` or to any ```<ViroNode>```. When Viro detects the image in the user's world, the content of the ```<ViroARImageMarker>``` will be rendered. The ```<ViroARImageMarker>``` will be continually tracked and will stay synchronized with the detected image. Below is an example of simple usage: [block:code] { "codes": [ { "code": "// In your render function, add an image marker that references the target\n<ViroARScene>\n <ViroARImageMarker target={\"targetOne\"} >\n <ViroBox position={[0, .25, 0]} scale={[.5, .5, .5]} />\n </ViroARImageMarker>\n</ViroARScene>\n\n// Outside of the render function, register the target\n\nViroARTrackingTargets.createTargets({\n \"targetOne\" : {\n source : require('./res/targetOne.jpg'),\n orientation : \"Up\",\n physicalWidth : 0.1 // real world width in meters\n },\n});\n\n", "language": "javascript" } ] } [/block] In the more complex example below, we do this with a "Black Panther" movie poster. Upon detecting the poster, we load a 3D object representing the Black Panther, and make him jump out of the poster. [block:code] { "codes": [ { "code": "// In your render function:\n\n<ViroARScene>\n <ViroAmbientLight color=\"#ffffff\" intensity={200}/>\n <ViroARImageMarker target={\"poster\"} \n onAnchorFound={this._onAnchorFound}\n pauseUpdates={this.state.pauseUpdates}>\n\n <ViroNode position={[0, -.1, 0]} scale={[0,0,0]} rotation={[-90, 0, 0]} \n dragType=\"FixedToWorld\" onDrag={()=>{}}\n animation={{name:\"scaleModel\", run:this.state.playAnim}}>\n \n <Viro3DObject onLoadEnd={this._onModelLoad}\n source={require('./res/blackpanther/object_bpanther_anim.vrx')}\n position={[0, -1.45, 0]}\n scale={[.9,.9,.9]}\n animation={{name:\"01\", run:true, loop:false, onFinish:this._onFinish}}\n type=\"VRX\" />\n </ViroNode>\n</ViroARImageMarker>\n \n// Outside the render function: \n \nViroARTrackingTargets.createTargets({\n poster : {\n source : require('./res/blackpanther.jpg'),\n orientation : \"Up\",\n physicalWidth : 0.6096 // real world width in meters\n }\n});", "language": "javascript" } ] } [/block] The full code for this sample is contained in the [Viro Samples](https://github.com/viromedia/viro/tree/master/js/ARPosterDemo) repository. The final result will look like this: [block:image] { "images": [ { "image": [ "https://files.readme.io/f56f163-viro_black_panther_marker_demo.gif", "viro_black_panther_marker_demo.gif", 600, 278, "#4b4547" ] } ] } [/block] [block:api-header] { "title": "Continuous Image Tracking (iOS 12 Only)" } [/block] ARKit 2.0 introduced a new API that enables continuous Image Tracking instead of simply image detection. This enables your marker to be tracked and followed smoothly as the user moves it around. This is exposed in [ViroARSceneNavigator's](doc:viroarscenenavigator) new **numberOfTrackedImages** property which takes in a number of images that should be tracked concurrently. For example, if this number is set to 3, then the first 3 images visible in a scene will be tracked. Even if there are 5 total ViroARImageMarkers only the first 3 will be tracked, if one marker leaves the view, then an untracked marker will then be tracked. Keep in mind that this number should be kept low as the higher the number, the worse the performance. [block:api-header] { "title": "Image Target Quality" } [/block] Google has released a tool (for OSX and Windows) to check the quality of Image Targets, check it out on their website here: https://developers.google.com/ar/develop/c/augmented-images/arcoreimg This tool provides a good baseline for Image Recognition on both iOS and Android