{"id":332,"date":"2009-11-25T16:47:42","date_gmt":"2009-11-25T16:47:42","guid":{"rendered":"https:\/\/www.shapeways.com\/blog\/archives\/332-interview-with-qi-pan-about-his-webcam-3d-scanner-proforma.html"},"modified":"2022-10-17T12:07:16","modified_gmt":"2022-10-17T16:07:16","slug":"interview-with-qi-pan-about-his-webcam-3d-scanner-proforma","status":"publish","type":"post","link":"https:\/\/www.shapeways.com\/blog\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma","title":{"rendered":"Interview with Qi Pan about his Webcam 3D scanner proForma"},"content":{"rendered":"

<\/p>\n

Qi Pan, a Cambridge University<\/a> researcher has developed proForma. ProForma is a tool that turns any normal webcam into a 3D scanner. I got to interview him and talk about his product, which if it worked could one of  the holy grails of mass customization. It would enable anyone to inexpensively turn things digital and then reproduce them.<\/p>\n

Joris Peels<\/b>: So how did you get started on this project?<\/b><\/p>\n

Qi Pan<\/b>: At the start of my PhD, I was interested in real-time 3D modeling of
\noutdoor scenes. However, several months in, I realised that current
\nprocessing power wasn’t enough to model outdoor scenes well (due to
\nocclusions, lack of texture, etc). Therefore I turned my attention to
\nsmaller objects, which would stand a better chance on current
\nhardware. With smaller objects though, they would always be sitting in
\nan environment, which you wouldn’t want to model, which led me to the
\nidea of using a fixed camera and separating the object using motion.
\nAll of the design choices made in the system were then tailored
\ntowards making everything as fast as possible, whilst still producing
\na reasonable output.<\/p>\n

How long did it take?<\/b><\/div>\n

The project as it stands has taken around a year and a half to
\ndevelop, although not all of that time was spent on development (time
\nwas also spent on publications and attending various conferences).<\/p>\n

What was hard to do?<\/b><\/div>\n

The hardest thing to do was to combine all of the system components
\ninto a real-time system. The problem with real-time is that if any one
\npart of the system is not working well, your system just doesn’t work
\nfull stop. Therefore you need to make sure all parts are well
\noptimised and producing the right output at the right time for the
\nother components. When designing each component, the utmost care had
\nto be taken to ensure that we were doing things as efficiently as
\npossible, using the best available algorithms (or inventing our own if
\nnone existed).<\/p>\n


\nHow does it work exactly?<\/b><\/div>\n

The system works in two stages.
\nThe first stage is a tracker, and uses the partial 3D model we’ve
\nconstructed to work out the position and orientation of the object
\nrelative to the camera. This stage also tracks the position of
\ninterest points (areas of high contrast change) in the images
\nframe-to-frame. After a significant enough motion is detected, a
\nkey-frame plus the interest point tracks are passed to the
\nreconstruction stage. Only interest points on the object are tracked
\nas there is a mathematical constraint on the motion of points on a
\nrigid object (based on Epipolar geometry).
\nThe reconstruction stage takes these feature tracks and triangulates
\n3D positions in order to form a cloud of points. This is then meshed
\nusing a 3D Delaunay tetrahedralisation. This however merely partitions
\nthe convex hull of the points into tetrahedra, so therefore we need to
\nemploy a carving algorithm to remove incorrect tetrahedra from
\nconcavities in the object. We formulated a very efficient
\nprobabilistic carving algorithm to achieve this, which allows us to
\nobtain the surface of the object based on the interest points we’ve
\nseen in each keyframe.
\nThis method requires a partial 3D model to track from, which isn’t
\navailable right at the start of reconstruction (but is later).
\nTherefore, our initialisation step differs slightly from normal
\noperation. We assume that at least part of the object falls within a
\nlarge circle at the centre of the image. We track interest points
\ninside this circle, and use rigid body motion constraints to ascertain
\nthe orientation and position of the object relative to the camera.
\nAmazingly, this is possible, even if we have no idea about the 3D
\npositions of the interest points we are tracking! The system then
\nworks as above once we have this initial orientation and position.<\/p>\n


\nBut, can I take a thing and then you will give me a mesh?<\/b><\/div>\n

Yes, as long as it is textured enough! The system is based on interest
\npoints, so the object must have enough areas of high contrast change.<\/p>\n


\nWhat are some of the limitations?<\/b>
\nThis system is of course only a first step in generic object
\nreconstruction, and as such has a few limitations. One limitation is
\nthe inability to model objects or parts of objects without enough<\/p>\n
texture. This is something we are working on – we are seeking to
\ncombine other cues to complement our interest point based approach.<\/div>\n

This approach can  in theory be applied to modeling entire scenes,<\/p>\n

but then we come up against the problems of the environment not being
\ntextured enough in areas, occlusion and needing more processing power.<\/div>\n

The technique as it stands can only be used to model rigid objects due
\nto the rigid body assumption being used for segmentation.<\/p>\n<\/div>\n

You will be working more on it in the future?<\/b><\/div>\n

Yes – we most certainly will! This project is more of a proof of
\nconcept and just the tip of the iceberg in terms of what we can
\nachieve.<\/p>\n

Will there be a tool that people can download?<\/b><\/div>\n

Yes – we’re currently working on releasing one soon.<\/p>\n

When?<\/b>
\nI’m currently porting the software to the newest libraries (which
\nunfortunately means reimplementing lots of stuff from scratch) but in
\na few months time we aim to release a linux-based demo which will
\nhopefully be followed by a windows based demo after that.<\/p>\n","protected":false},"excerpt":{"rendered":"

Qi Pan, a Cambridge University researcher has developed proForma. ProForma is a tool that turns any normal webcam into a…<\/p>\n","protected":false},"author":6,"featured_media":3486,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[481,478],"tags":[480,483,485,484,559,560,479,944],"class_list":["post-332","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-shapeways","category-community","tag-4-art","tag-7-contests","tag-9-gadget-lab","tag-8-gadgets","tag-inspiration","tag-software","tag-3-tutorials","tag-whats-hot"],"thumbnail":false,"yoast_head":"\nInterview with Qi Pan about his Webcam 3D scanner proForma - Shapeways Blog<\/title>\n<meta name=\"description\" content=\"Qi Pan, a Cambridge University researcher has developed proForma. ProForma is a tool that turns any normal webcam into a 3D scanner. I got to interview\" \/>\n<meta name=\"robots\" content=\"noindex, follow\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"joris\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#article\",\"isPartOf\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma\"},\"author\":{\"name\":\"joris\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/person\/938b4c3f5430a23320b54d0ff1d4bba0\"},\"headline\":\"Interview with Qi Pan about his Webcam 3D scanner proForma\",\"datePublished\":\"2009-11-25T16:47:42+00:00\",\"dateModified\":\"2022-10-17T16:07:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma\"},\"wordCount\":918,\"commentCount\":8,\"publisher\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#primaryimage\"},\"thumbnailUrl\":\"\",\"keywords\":[\"Art\",\"Contests\",\"Gadget Lab\",\"Gadgets\",\"inspiration\",\"software\",\"Tutorials\",\"What's Hot\"],\"articleSection\":[\"Shapeways\",\"The Community\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma\",\"url\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma\",\"name\":\"Interview with Qi Pan about his Webcam 3D scanner proForma - Shapeways Blog\",\"isPartOf\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#primaryimage\"},\"image\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#primaryimage\"},\"thumbnailUrl\":\"\",\"datePublished\":\"2009-11-25T16:47:42+00:00\",\"dateModified\":\"2022-10-17T16:07:16+00:00\",\"description\":\"Qi Pan, a Cambridge University researcher has developed proForma. ProForma is a tool that turns any normal webcam into a 3D scanner. I got to interview\",\"breadcrumb\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#primaryimage\",\"url\":\"\",\"contentUrl\":\"\",\"width\":510,\"height\":112},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/prodshapeblog.wpenginepowered.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Interview with Qi Pan about his Webcam 3D scanner proForma\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#website\",\"url\":\"https:\/\/prodshapeblog.wpenginepowered.com\/\",\"name\":\"Shapeways Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/prodshapeblog.wpenginepowered.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#organization\",\"name\":\"Shapeways Blog\",\"url\":\"https:\/\/prodshapeblog.wpenginepowered.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.shapeways.com\/blog\/wp-content\/uploads\/2019\/05\/shapeways-blog.png\",\"contentUrl\":\"https:\/\/www.shapeways.com\/blog\/wp-content\/uploads\/2019\/05\/shapeways-blog.png\",\"width\":943,\"height\":121,\"caption\":\"Shapeways Blog\"},\"image\":{\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"http:\/\/facebook.com\/shapeways\",\"https:\/\/x.com\/shapeways\",\"http:\/\/instagram.com\/shapeways\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/person\/938b4c3f5430a23320b54d0ff1d4bba0\",\"name\":\"joris\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9e438617ae718efd30ac443b071fd9793ea032327fb5779c3e2d3e92f727eaa0?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9e438617ae718efd30ac443b071fd9793ea032327fb5779c3e2d3e92f727eaa0?s=96&d=mm&r=g\",\"caption\":\"joris\"},\"sameAs\":[\"http:\/\/\"],\"url\":\"https:\/\/www.shapeways.com\/blog\/author\/JorisPeels\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Interview with Qi Pan about his Webcam 3D scanner proForma - Shapeways Blog","description":"Qi Pan, a Cambridge University researcher has developed proForma. ProForma is a tool that turns any normal webcam into a 3D scanner. I got to interview","robots":{"index":"noindex","follow":"follow"},"twitter_misc":{"Written by":"joris","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#article","isPartOf":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma"},"author":{"name":"joris","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/person\/938b4c3f5430a23320b54d0ff1d4bba0"},"headline":"Interview with Qi Pan about his Webcam 3D scanner proForma","datePublished":"2009-11-25T16:47:42+00:00","dateModified":"2022-10-17T16:07:16+00:00","mainEntityOfPage":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma"},"wordCount":918,"commentCount":8,"publisher":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#organization"},"image":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#primaryimage"},"thumbnailUrl":"","keywords":["Art","Contests","Gadget Lab","Gadgets","inspiration","software","Tutorials","What's Hot"],"articleSection":["Shapeways","The Community"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#respond"]}]},{"@type":"WebPage","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma","url":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma","name":"Interview with Qi Pan about his Webcam 3D scanner proForma - Shapeways Blog","isPartOf":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#primaryimage"},"image":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#primaryimage"},"thumbnailUrl":"","datePublished":"2009-11-25T16:47:42+00:00","dateModified":"2022-10-17T16:07:16+00:00","description":"Qi Pan, a Cambridge University researcher has developed proForma. ProForma is a tool that turns any normal webcam into a 3D scanner. I got to interview","breadcrumb":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#primaryimage","url":"","contentUrl":"","width":510,"height":112},{"@type":"BreadcrumbList","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/interview-with-qi-pan-about-his-webcam-3d-scanner-proforma#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/prodshapeblog.wpenginepowered.com\/"},{"@type":"ListItem","position":2,"name":"Interview with Qi Pan about his Webcam 3D scanner proForma"}]},{"@type":"WebSite","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#website","url":"https:\/\/prodshapeblog.wpenginepowered.com\/","name":"Shapeways Blog","description":"","publisher":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/prodshapeblog.wpenginepowered.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#organization","name":"Shapeways Blog","url":"https:\/\/prodshapeblog.wpenginepowered.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.shapeways.com\/blog\/wp-content\/uploads\/2019\/05\/shapeways-blog.png","contentUrl":"https:\/\/www.shapeways.com\/blog\/wp-content\/uploads\/2019\/05\/shapeways-blog.png","width":943,"height":121,"caption":"Shapeways Blog"},"image":{"@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/logo\/image\/"},"sameAs":["http:\/\/facebook.com\/shapeways","https:\/\/x.com\/shapeways","http:\/\/instagram.com\/shapeways"]},{"@type":"Person","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/person\/938b4c3f5430a23320b54d0ff1d4bba0","name":"joris","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/prodshapeblog.wpenginepowered.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/9e438617ae718efd30ac443b071fd9793ea032327fb5779c3e2d3e92f727eaa0?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9e438617ae718efd30ac443b071fd9793ea032327fb5779c3e2d3e92f727eaa0?s=96&d=mm&r=g","caption":"joris"},"sameAs":["http:\/\/"],"url":"https:\/\/www.shapeways.com\/blog\/author\/JorisPeels"}]}},"featured_image_src":null,"featured_image_src_square":null,"author_info":{"display_name":"joris","author_link":"https:\/\/www.shapeways.com\/blog\/author\/JorisPeels"},"_links":{"self":[{"href":"https:\/\/www.shapeways.com\/blog\/wp-json\/wp\/v2\/posts\/332","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shapeways.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shapeways.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shapeways.com\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shapeways.com\/blog\/wp-json\/wp\/v2\/comments?post=332"}],"version-history":[{"count":0,"href":"https:\/\/www.shapeways.com\/blog\/wp-json\/wp\/v2\/posts\/332\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.shapeways.com\/blog\/wp-json\/wp\/v2\/media?parent=332"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shapeways.com\/blog\/wp-json\/wp\/v2\/categories?post=332"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shapeways.com\/blog\/wp-json\/wp\/v2\/tags?post=332"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}