Jamaica Tallawahs Digital Jersey: Powered by Dress Segmentation technology

I and our team was researching on 𝗱𝗿𝗲𝘀𝘀 𝘀𝗲𝗴𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝘁𝗼 𝗱𝗲𝘃𝗲𝗹𝗼𝗽 𝘀𝗼𝗺𝗲 𝘀𝗽𝗲𝗰𝗶𝗮𝗹 𝗔𝗥 𝗲𝗳𝗳𝗲𝗰𝘁𝘀 𝗮𝗿𝗼𝘂𝗻𝗱 𝗨𝘀𝗲𝗿’𝘀 𝗼𝘂𝘁𝗳𝗶𝘁. We converted the 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 T-𝘀𝗵𝗶𝗿𝘁 𝗮𝘀 𝘁𝗵𝗲 𝗰𝗮𝗻𝘃𝗮𝘀 𝗳𝗼𝗿 𝗔𝗥 𝗲𝗳𝗳𝗲𝗰𝘁𝘀 in order to deliver a unique digital jersey experience unlike the traditional way of overlaying 3d Model of jerseys on top of the users.

So during the campaign, Jamaica Tallawahs’s fans can turn their physical outfit into a Jamaica jersey and use the t.shirt as a canvas for Jamaica themed immersive AR effects.

We used SparkAR to develop this cloth/dress segmentation capability. Though sparkAR doesn’t have built-in dress segmentation capability, 𝗪𝗲 𝗳𝘂𝘀𝗲𝗱 𝗳𝗲𝘄 𝗼𝘁𝗵𝗲𝗿 𝘀𝗽𝗮𝗿𝗸𝗔𝗥’𝘀 𝗔𝗜 𝘁𝗿𝗮𝗰𝗸𝗶𝗻𝗴 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 𝘁𝗼 achieve 𝘁𝗵𝗶𝘀 𝗗𝗿𝗲𝘀𝘀 𝘀𝗲𝗴𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝘆.

To make this T-shirt experience more immersive and realistic we have to 

  ⭐ 𝘀𝗲𝗮𝗺𝗹𝗲𝘀𝘀𝗹𝘆 𝗯𝗹𝗲𝗻𝗱 𝗔𝗥 𝗲𝗳𝗳𝗲𝗰𝘁𝘀 (Jamaica themed effect) around 𝘁𝗵𝗲 𝘂𝘀𝗲𝗿'𝘀 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗰𝗹𝗼𝘁𝗵𝗶𝗻𝗴 instead of just overlaying a 3D model of the t.shirt on top of the user all the time. 

  ⭐ 𝗔𝗻𝗰𝗵𝗼𝗿 𝟯𝗗 𝘀𝘁𝗶𝗰𝗸𝗲𝗿𝘀 (JT & its sponsors 𝗹𝗼𝗴𝗼𝘀) 𝘁𝗼 𝘁𝗵𝗲 T-𝘀𝗵𝗶𝗿𝘁 instead of just masking it. 

Our filter experience segments dress and augment cool AR effects and anchor sponsored logos/3D sticks on top of the t.shirt that the user wears. 

Some key technical progress we made during development were

✔️ Segmenting clothing of user and augmenting cool AR effect on top of the physical T-shirt

✔️ Dress Segmentation alone 𝐰𝐨𝐧’𝐭 𝐠𝐢𝐯𝐞 𝐮𝐬 𝐭𝐡𝐞 𝐩𝐨𝐬𝐞 𝐨𝐟 𝐭𝐡𝐞 𝐮𝐬𝐞𝐫, so we found a way to anchor logos to shirts instead of just masking them. We found a solution similar to upper body tracking to achieve this.

Now users can try a dynamic digital jersey experience, this experience is perfect for reels, So they can use that for their Instagram reels.