• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

derenlei/Unity_Detection2AR: Localize 2D image object detection in 3D Scene with ...

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称(OpenSource Name):

derenlei/Unity_Detection2AR

开源软件地址(OpenSource Url):

https://github.com/derenlei/Unity_Detection2AR

开源编程语言(OpenSource Language):

C# 100.0%

开源软件介绍(OpenSource Introduction):

Unity_Detection2AR

A simple solution to incorporate object localization into conventional computer vision object detection algorithms.

IDEA: There aren't that many open source real-time 3D object detection. This is an example of using "more popular" 2D object detection and then localize it with a few feature points. It uses recently released Barracuda for object detection and ARFoundation for AR. It works both on iOS and Android devices.

Currently supports tiny Yolo2 and 3.

demo

Requirements

"com.unity.barracuda": "1.0.3",
"com.unity.xr.arfoundation": "4.0.8",
"com.unity.xr.arkit": "4.0.8",
"com.unity.xr.arcore": "4.0.8"

Usage

It is developed in Unity 2020.2.1 and requires product ready Barracuda with updated AR packages. The preview Barracuda versions seems unstable and may not work.

  • Open the project in Unity (Versions > 2019.4.9).
  • In Edit -> Player Settings -> Other XR Plug-in Management, make sure Initialize XR on Startup and Plug-in providers are marked to enable ARCamera.\
  • From Inspector Scene: Detect -> Game Object: Camera Image -> Script: Phone AR Camera, choose Selected_detector to either Yolo2_tiny or Yolo3_tiny(default).
  • Make sure that Detector has ONNX Model file and Labels file set.
  • For Android, check the Minimum API Level at Project Settings -> Player -> Others Settings -> Minimum API Level. it requires at least Android 7.0 'Nougat' (API Level 24).
  • For Android, also enable Auto Graphics API. See Issue
  • In File -> Build settings choose Detect and hit Build and run.
  • For IOS, fix team setting in Signing & Capabilities.

Detection Model

We currently support Yolo version 2 (tiny) and Yolo version 3 (tiny). Example models are in Assets/Models/.

yolov3-tiny-416.onnx is trained on COCO dataset.

yolov2-tiny-food-freeze.onnx is trained on FOOD100 dataset through darknet. A good example of the training tool is here. Ideally, it can detect 100 categories of dishes.

Image

Use Your Own Model

  1. Convert your model into the ONNX format. If it is trained through Darknet, convert it into frozen tensorflow model first, then ONNX.
  2. Upload the model and label to Assets/Models. Use inspector to update your model settings in Scene: Detect -> Game Object: Detector Yolo2-tiny / Detector Yolo3-tiny. Update anchor info in the DetectorYolo script here or here.

Acknowledgement




鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
kburfi/EstimoteLocate: Indoor localization app for Estimote Beacons发布时间:2022-08-16
下一篇:
achimr/m4loc: Moses for Localization发布时间:2022-08-16
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap