Sony announces the "Mocopi" - A dedicated motion capture device for use in VR, AR and the Metaverse

[Meag]

Summary

Sony has officially announced a new piece of hardware named "Mocopi" through it's Japanese YouTube channel.

 

Mocopi is a set of six small tracking tags that you attach on various parts of your body, one for your head, hips, both wrist and both ankles to allow you to experience full body tracking in a variety of applications in VR, AR and the Metaverse. Currently, they connect through Bluetooth to an Android or iOS device to provide motion data, services such as Unity are then able to take advantage of this to replicate full body tracking setup similar to that of the HTC Vive body trackers, although it's unclear at this time of the accuracy of the tracking in comparison to those used by HTC or other products, such as the Slime VR body trackers. The price is reported to be around the $360 USD or equivalent mark.

 

Image source: https://www.bloomberg.com/news/articles/2022-11-29/sony-s-new-metaverse-bet-360-wearable-that-captures-your-moves

 

Quotes

Quote

Sony has announced Mocopi, a portable and lightweight motion capture system that can be used to control a digital avatar in real time within metaverse applications like VR Chat. The kit consists of six button-like tracking tags — one for your head, hip, both ankles, and both wrists — that use Bluetooth to pair with an Android or iOS smartphone app to input motion data to compatible services like Unity.

 

The Mocopi sensors measure 32mm (around 1.2 inches) in diameter and weigh just 8g, making them lighter than an Apple AirTag despite being of a similar size. The system is portable thanks to being completely wireless and not requiring a base station, and the trackers can be fully charged in around 90 minutes via the USB-C charging case. Sony claims Mocopi can achieve up to 10 hours of battery life, though this will depend on the usage environment.

Quote Source: https://www.theverge.com/2022/11/30/23485732/sony-mocopi-motion-capture-metaverse-vr-avatar-announcement-price

 

My thoughts

Assuming the tracking is up to snuff and Sony allows for easy integration to existing applications such as VR Chat, BeatSaber and the like, I can imagine these getting quite popular. They clearly are interested in supporting the use of these for "Anime" styled avatars, so I can imagine them being a shoe in for a go to VRChat / VTuber accessory for the diehards out there. The reported $360 USD price tag will put many off, especially those with a launch price Oculus Meta Quest 2, as the price eclipse the previously $300 64gb base model. I personally would love to try a demo, but it's unclear what Sony's plans are in relation to world wide availability, but given that they've only showcased this on their Japanese YouTube channel, it suggests it may be more difficult to get your hands on in the west.

 

Sources

https://www.theverge.com/2022/11/30/23485732/sony-mocopi-motion-capture-metaverse-vr-avatar-announcement-price

https://www.bloomberg.com/news/articles/2022-11-29/sony-s-new-metaverse-bet-360-wearable-that-captures-your-moves

 

 

[porina]

Really interested to see how this goes. As long as it works well, that price I don't feel is a barrier to anyone at all serious as a decent model would be WAY more expensive. Wonder how it interfaces to existing software. Edit: that's from a VTuber perspective. I have no interest in VRChat.

[Kisai]

54 minutes ago, Meag said:

Summary

Sony has officially announced a new piece of hardware named "Mocopi" through it's Japanese YouTube channel.

 

Mocopi is a set of six small tracking tags that you attach on various parts of your body, one for your head, hips, both wrist and both ankles to allow you to experience full body tracking in a variety of applications in VR, AR and the Metaverse. Currently, they connect through Bluetooth to an Android or iOS device to provide motion data, services such as Unity are then able to take advantage of this to replicate full body tracking setup similar to that of the HTC Vive body trackers, although it's unclear at this time of the accuracy of the tracking in comparison to those used by HTC or other products, such as the Slime VR body trackers. The price is reported to be around the $360 USD or equivalent mark.

 

Image source: https://www.bloomberg.com/news/articles/2022-11-29/sony-s-new-metaverse-bet-360-wearable-that-captures-your-moves

 

Quotes

 

My thoughts

Assuming the tracking is up to snuff and Sony allows for easy integration to existing applications such as VR Chat, BeatSaber and the like, I can imagine these getting quite popular. They clearly are interested in supporting the use of these for "Anime" styled avatars, so I can imagine them being a shoe in for a go to VRChat / VTuber accessory for the diehards out there. The reported $360 USD price tag will put many off, especially those with a launch price Oculus Meta Quest 2, as the price eclipse the previously $300 64gb base model. I personally would love to try a demo, but it's unclear what Sony's plans are in relation to world wide availability, but given that they've only showcased this on their Japanese YouTube channel, it suggests it may be more difficult to get your hands on in the west.

 

Sources

https://www.theverge.com/2022/11/30/23485732/sony-mocopi-motion-capture-metaverse-vr-avatar-announcement-price

https://www.bloomberg.com/news/articles/2022-11-29/sony-s-new-metaverse-bet-360-wearable-that-captures-your-moves

 

 

It has never been an issue to get stuff from Japan if you want it bad enough. The issue is the software. Japanese vtubing software tends to have english UI's, but they're still overcomplicated messes.

 

Presently, if you want to be a 3D Vtuber, your options are:

VSeeFace (No longer works as of Nvidia 526 drivers)

3Tene

VUP

 

Of these software products you need at least the following hardware:

- a webcam (for the worst experience) for face, body or hand tracking  (pick ONE)

- an iPhone X or better (for the best experience) for face tracking

- motion trackers (most common are vive trackers) for body tracking

- finger-hand tracking (done via a separate specialized camera unit, eg leapmotion)

 

No HMD. You're not going to wear a VR HMD unless you're playing VRChat, or Beatsaber. You're not going to buy the Meta Quest Pro to be a vtuber, it just doesn't make sense to spend that much money for what amounts to just a head tracker that can't even track the lip/jaw and eye/brow movement.

 

Vtubers who have money to burn on a full body setup will skip the trackers and go straight to a mocap suit. The problem with going this route is that while the suit itself might actually be fairly cheap itself, the software is not, and mocap software is some of the worst subscription BS there is. At least two English-speaking vtubers are known to use full body setups with mocap suits (Eg CodeMiko and Project Melody), and even then, most vtubers would rather use more detailed Live2D models than janky 3D models with full body when they spend most of their time sitting in a chair. 3D tracking is 

 

What would be the real break through piece of tech would be combining trackers with a stereo webcam that can track the face, body and fingers. Because, relying on Apple to not change the face tracking is still a gamble (and Apple has degraded the face tracking on iOS 15.x) Like Intel's RealSense could have had traction here, but they've abandoned the product.

 

A common joke in the Vtuber community is having people go "Why don't you use the kinect?", which the answer is "What Kinect?", As Microsoft discontinued it long before anyone got into Vtubing, and the Kinect never actually did good face tracking (20 params vs 52 in ARKit,) only some rudimentary body tracking. The Azure Kinect is essentially unusable.

 

Pretty much among the vtuber community, If anyone uses trackers, they use the vive trackers, and they're not great, because they break easily. We're kind of in this neutral position where the price of trackers has gone down, but the quality hasn't gone up, and the software needed to use trackers is still not great. The problem here, is that these trackers use the phone, which means to use these trackers to use with face tracking would require a second mobile phone, which is a non-starter for most people. 

 

The rokoko mocap suit costs $2500, the Perception Neutron suit is $4000-7500, and these are so extremely far out of the cost range for all but maybe 10 vtubers in the world. A $350 set of 6 trackers is cheap compared to $160 per tracker for the vive trackers.

 

But ultimately if you're seriously into vtubing, you need to be willing to spend $5000 on the model alone. Most people doing vtubing right now are either:

 

2D using Vtube Studio and an iPhone

2D using Vtube Studio a webcam (with or without nvidia RTX broadcast)  

3D using VSeeFace

2D or 3D using 3Tene Pro

2D using PrprLive

2D or 3D using VUP

 

If you're unwilling to spend money on the model at some point, then buying the hardware that has no other purpose is pointless. Sure, you can use the VR trackers for VR, but if you didn't buy the VR HMD, the trackers have no other actual use. Sure you can use it to maybe record motion data for video games, if you're developing a video game. But most of the uses of full-body tracking involve puppeting an existing model you have the rights to use. So unless you're willing to spend money on that (or design one yourself) buying "just the trackers" doesn't have much utility.

 

[wanderingfool2]

1 hour ago, Kisai said:

It has never been an issue to get stuff from Japan if you want it bad enough. The issue is the software. Japanese vtubing software tends to have english UI's, but they're still overcomplicated messes.

With integration with Unity and by the sounds of it being able to pull the data then I could see this having quite a bit of practical applications (vtubing aside).

 

Just inherently you could use it to help with things such as blender animations and such...and when you introduce something simpler like this, you could open up to a lot more open source applications for things such as vtuber.....or in general using it for different controls.  Honestly, if they could get the pricing to more like $150 I could see uses for this product (at the current price probably not).

[porina]

2 hours ago, Kisai said:

But ultimately if you're seriously into vtubing, you need to be willing to spend $5000 on the model alone. Most people doing vtubing right now are either:

 

2D using Vtube Studio and an iPhone

2D using Vtube Studio a webcam (with or without nvidia RTX broadcast)  

2D using PrprLive

I've not looked recently but a decent enough L2D model (art and rigging) doesn't need to cost 5k. If you go for a well established artist and rigger then maybe, but there will be up and coming people in the space looking to make a name for less. Mine was around $300 for both, but my creator did admit they were learning still and it was an interesting process. The last time I looked they charged $900 just for rigging.

 

I find prprlive + webcam works well enough. My model isn't suited to vtube studio unless there's some option in there I haven't found yet to limit movement within working ranges.

 

I was sad when Kinect was killed off, as I feel that had potential combined with VR space but they were too temporally separated. Multi-camera image tracking might be viable with sufficient processing, but direct physical tracking seems less likely to be distracted.

 

One more potential use case might be MMD like uses. Lots of models available. Getting new motion data in, less so.

 

58 minutes ago, wanderingfool2 said:

Honestly, if they could get the pricing to more like $150 I could see uses for this product (at the current price probably not).

I really think the price is fine for those who seriously have a use for it. It only seems high if you want a toy to play with.

[Kisai]

3 minutes ago, porina said:

I've not looked recently but a decent enough L2D model (art and rigging) doesn't need to cost 5k. If you go for a well established artist and rigger then maybe, but there will be up and coming people in the space looking to make a name for less. Mine was around $300 for both, but my creator did admit they were learning still and it was an interesting process. The last time I looked they charged $900 just for rigging.

The cost to get into vtubing is 0$ if you have an iphone. But you still need a rigged model, which can range from free models to paying lots of money. 

 

The point of the bolded line is "if you aren't prepared to spend significant money on the model, then why would you spend that much on the hardware?" A Rokoko suit is a better "investment" than trackers, but if your only other non-vtuber use case for owning the suit is VRChat, then you are spending a lot of money on hardware that has no utility over trackers. Trackers are more suited for games precisely because with the exception of VRChat, the game isn't responding to 19 sensors, and you don't have tactile feedback around those sensors.  With VRChat, other people can see you, so better animations are better, but we're kidding ourselves if anyone is going to be playing VRChat with a mocap suit when the actual model rigging doesn't account for those sensors, only the 5 trackers (ankles, wrists, and hips.) Wrists tend duplicate tracking with the controllers.

 

 

3 minutes ago, porina said:

I find prprlive + webcam works well enough. My model isn't suited to vtube studio unless there's some option in there I haven't found yet to limit movement within working ranges.

If your model is rigged with the 52 ARKit parameters it can be used in VTS, 3tene and prprlive. Generally VTS is geared better for western vtubers  on twitch (or youtube), where as prprlive is geared for billibilli. Prprlive also requires an account for certain features. VTS is geared around the privacy of the vtuber and has features explicitly to prevent the display of the vtuber's face, ip address, and file system.

 

3 minutes ago, porina said:

I was sad when Kinect was killed off, as I feel that had potential combined with VR space but they were too temporally separated. Multi-camera image tracking might be viable with sufficient processing, but direct physical tracking seems less likely to be distracted.

 

One more potential use case might be MMD like uses. Lots of models available. Getting new motion data in, less so.

 

I really think the price is fine for those who seriously have a use for it. It only seems high if you want a toy to play with.

MMD really took off when the first gen Kinect 360 had a third party OpenNI implementation. Microsoft hasn't put much effort into developing Kinect, and seems to care more about business uses of the Azure Kinect that doesn't have much application for vtubing or games.

 

So I feel that cheap trackers is a viable way to get people into 3D vtubing (Eg VRoid models) but the basic requirements for face tracking remains "iphone only" or having an expensive RTX GPU with a reasonable webcam. Existing cpu-only tracking is still substandard to the iphone.

 

[porina]

26 minutes ago, Kisai said:

The point of the bolded line is "if you aren't prepared to spend significant money on the model, then why would you spend that much on the hardware?"

That wasn't obvious to me. The line stood alone so I took it to be intended universally.

 

26 minutes ago, Kisai said:

If your model is rigged with the 52 ARKit parameters it can be used in VTS, 3tene and prprlive. Generally VTS is geared better for western vtubers  on twitch (or youtube), where as prprlive is geared for billibilli.

I got in relatively early. My model was actually commissioned for use with FaceRig, just before devs killed it, and then killed themselves releasing Animaze that no one wanted to pay the sub for. I tried prprlive since my creator suggested it and it worked. VTS didn't exist when my model completed Jan 2021, coming out some months later. I paid the one-time fee for prprlive improved tracking DLC and don't need anything other than Steam account. I can't see how prprlive can leak any info if configured correctly, that is sharing through the virtual camera interface and not doing something stupid like using the app screen itself.

 

Anyway, been thinking of reinventing myself to broaden away from a specific game, and if body tracking becomes more accessible then I might consider a 3D model next instead of L2D. I had wondered what it would take to get some virtual 2D arms, but at that point, why not other parts too?

[wanderingfool2]

3 hours ago, porina said:

I really think the price is fine for those who seriously have a use for it. It only seems high if you want a toy to play with.

Oh yea, it is currently a good pricing...but I think if it were to hit the $150 mark you would have it at a level that more general consumers might purchase it and tinker around with it (because it genuinely could be a fun thing to do)

[Thaldor]

5 hours ago, wanderingfool2 said:

Oh yea, it is currently a good pricing...but I think if it were to hit the $150 mark you would have it at a level that more general consumers might purchase it and tinker around with it (because it genuinely could be a fun thing to do)

At $150 mark it would start to feel like a dollar store product, way too cheap to be actually good and there must be some kind of hook. Or like the Lenovo Star Wars Jedi Challenge AR headset from few years ago that wasn't that cheap but it boasted on VR/AR and was cheaper than anything else then and the trick was that's all you got, $200 and it was at most couple hours of fun and after the app stops working, it's paperweight.

 

At $360 it's at least believable product that there actually is sensors in it and it isn't just to monetize on seconds whim. At that price it actually does compete in the "consumer mocap" market with stuff like just janky webcams to Microsoft Kinect to Vive Trackers. If it actually works it's a bit more than Vive Trackers with tiny bit more price (if we go with VR stuff you already have hands and head tracked so you need hip and legs and with Tracker 3.0's that is going to be around $350). It probably won't be close to Perception Neuron which is there $3,000 to $7,500 prices depending what you want but those are professional mocap stuff and I feel like that comparison would be same as comparing Kinect to OptiTrack system and actually expect Kinect to deliver even half what 6/8 cam OptiTrack can deliver at 100-times the price.

[wanderingfool2]

4 minutes ago, Thaldor said:

At $150 mark it would start to feel like a dollar store product, way too cheap to be actually good and there must be some kind of hook

Well I mean, there are 6 "devices" which are effectively all the same.  An accelerometer/gyroscope combo apparently can run sub $5, so at bulk pricing it's likely less, plus a bluetooth module.  So I wouldn't be surprised if they could manufacture each device for sub $10...making the cost about $60.  Just a guess though...but based on what is shown I wouldn't be surprised if it only costs $60 to manufacture.

 

I'm just saying at $150 it starts getting into the realm of more common day people maybe willing to try it and tinker with.  At $300 I think it's in a price range that a lot less people would want to tinker as it is something that unless you have a use for it's harder to justify buying.

[Thaldor]

1 hour ago, wanderingfool2 said:

Well I mean, there are 6 "devices" which are effectively all the same.  An accelerometer/gyroscope combo apparently can run sub $5, so at bulk pricing it's likely less, plus a bluetooth module.  So I wouldn't be surprised if they could manufacture each device for sub $10...making the cost about $60.  Just a guess though...but based on what is shown I wouldn't be surprised if it only costs $60 to manufacture.

 

I'm just saying at $150 it starts getting into the realm of more common day people maybe willing to try it and tinker with.  At $300 I think it's in a price range that a lot less people would want to tinker as it is something that unless you have a use for it's harder to justify buying.

But with those <$10 parts is there anything to tinker with or is it just the same as the Lenovo Jedi Challenge, made so cheaply it can barely do what it is made to do and even then downgraded to the bare minimum? As in the Jedi Challenge was basicly a bit more complicated than the Nintendo 3DS AR cards or any "hidden image" AR that just looks for an image/object and renders whatever model on top of that, IIRC even the sabre worked exactly the same, there was 0 gyroscopes or accelerometers, it was all just 2 cameras, screen, some buttons, BT circuit and batteries. Calling it AR headset was almost the same as calling the View-Master a VR headset. It was all just the software on your phone and in the actual product there was nothing to tinker with, it was just a gimmick toy to capitalize on seconds whim.

 

Looking at those, they could be done way cheaper than $60 even. Just write a software that uses the camera and focuses on the colored dots and vóla, you have basicly made OptiTrack that uses one camera. It's going to suck as much as any webcam based face/body-tracking software but it's cheap and we get to the point that is it even worth the little it costs? With more precise sensors, higher grade BT circuit and all that the price will rise and fast but they also will be more than a gimmick toy with hardly anything to tinker with because it is more than just the software that manages to make some sense from the little the cheapest hardware can provide.

[PeachGr]

I hope that it works with windows and exports to fbx, so that we ll have an option for cheap and good mocap suits 

[soldier_ph]

"Now you too can be an anime Girl in the Metaverse with accurate Body Tracking"

[Kinda Bottlenecked]

Looks good. The lack of hand tracking is still sad but seeing how the tracking nodes work they might release a new module down the line. It'll probably be a cheap alternative to mocap too. 

 

Since there's only ankle and hand nodes, it seems that they're using IK on the rig in the app.