ETH Zurich :
Computer Science :
Pervasive Computing :
Distributed Systems :
Research :
Mobile Interaction
Mobile Interaction touching Augmented Reality and Context Awareness
A Research Topic of the Distributed Systems Group
A large part of the information needs in everyday settings is directly related
to real-world entities, like objects, places, and people. Attaching
information to those entities thus has many benefits for the quality of
interaction between humans and their environment. For example, an object could
provide users with its own object-history, which could be used to infer object
relationships and interaction contexts in which the physical artifact has taken
part. By connecting physical artifacts to information in the virtual world,
such "physical hyperlinks" can serve as visible entry points into the
information infrastructure, thus increasing accessibility and visibility of a
virtual environment from within the physical world.
Narrowing the gap between the physical world surrounding the user and the
virtual world requires a kind of "bridge" between entities in the
real world and the associated information in the virtual world. Interaction
with the virtual aspects of an entity requires a device which is able to detect
the entity and display related information. Such a "symbolic magnifying
glass" allows, for example, to make object relations visible, to access
virtual functions of an object, and to virtually zoom into and out of an object
in order to get to know the object details or the object's context,
respectively. Physical objects thus become "interactive" in a way
which formerly was only possible with purely virtual, computer-based objects.
See also the following related items:
Participating Researchers
Gábor Sörös, Christian Flörkemeier (until 2014), Christof Roduner (until 2014), Robert Adelmann (until 2012), Philipp Bolliger (until 2011), Steve Hinske (until 2010), Michael Rohs (until 2005)
Selected Publications
See the Publications of the Distributed Systems Group page for a full listing of our publications.
- Valentin Vasiliu, Gábor Sörös
Coherent rendering of virtual smile previews with fast neural style transfer.
Proceedings of the IEEE Symposium on Mixed and Augmented Reality, ISMAR'19, Beijing, China. October 2019
Abstract, BibTeX, Paper (.pdf)
Other files: Supplement, Talk
- Gábor Sörös, Julia Giger, Jie Song
Demo: Solfège Hand Sign Recognition with Smart Glasses.
Live demo and poster at the First International Workshop on Egocentric Perception, Interaction, and Computing (EPIC 2016), in conjuction with the European Conference on Computer Vision (ECCV 2016), Amsterdam, the Netherlands. October 2016
Abstract, BibTeX, Paper (.pdf)
Other files: Poster, Video, Video
- Gábor Sörös, Marcel Lancelle, Nicolas Degen, Roland Mörzinger
Demo: Mobile Augmented Reality for Dentistry: Virtual Try-On in Live 3D.
Live demo and poster at the European Conference on Computer Vision (ECCV 2016), Amsterdam, the Netherlands. October 2016
Abstract, BibTeX
- Frederik Hermans, Liam McNamara, Gábor Sörös, Christian Rohner, Thiemo Voigt, Edith Ngai
Focus: Robust Visual Codes for Everyone.
Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys 2016), Singapore, Singapore. pp. 319-332, June 2016
Abstract, BibTeX, Paper (.pdf)
Other files: Talk
- Gábor Sörös
Wearable Barcode Scanning: Advancements in Visual Code Localization, Motion Blur Compensation, and Gesture Control.
PhD thesis No. 23464, ETH Zurich, Zurich, Switzerland, 2016
Abstract, BibTeX, Paper (.pdf)
Other files: Talk, Supplement
- Gábor Sörös, Stephan Semmler, Luc Humair, Otmar Hilliges
Fast Blur Removal for Wearable QR Code Scanners.
Proceedings of the 19th International Symposium on Wearable Computers (ISWC 2015), Osaka, Japan. September 2015
Abstract, BibTeX, Paper (.pdf)
Other files: Supplement, Talk, Video
- Jie Song, Gábor Sörös, Fabrizio Pece, Otmar Hilliges
Demo: Estimation of 3D Hand Position and Gestures on Unmodified Wearable Devices.
Live demo and poster at the IEEE Workshop on Observing and Understanding Hands in Action (HANDS 2015), in conjunction with the 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), Boston, MA, USA. IEEE, June 2015
Abstract, BibTeX, Paper (.pdf)
Other files: Poster
- Gábor Sörös, Severin Münger, Carlo Beltrame, Luc Humair
Multiframe Visual-Inertial Blur Estimation and Removal for Unmodified Smartphones.
Proceedings of the 23rd International Conference on Computer Graphics, Visualization and Computer Vision (WSCG 2015), Plzen, Czech Republic. June 2015
Abstract, BibTeX, Paper (.pdf)
Other files: Supplement, Talk
- Jie Song, Fabrizio Pece, Gábor Sörös, Marion Koelle, Otmar Hilliges
Joint Estimation of 3D Hand Position and Gestures from Monocular Video for Mobile Interaction.
Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 2015), Seoul, South Korea. ACM, April 2015
Abstract, BibTeX, Paper (.pdf)
Other files: Talk, Video
- Simon Mayer, Yassin N. Hassan, Gábor Sörös
A Magic Lens for Revealing Device Interactions in Smart Environments.
Proceedings of the SIGGRAPH Asia 2014 Symposium on Mobile Graphics and Interactive Applications (MGIA 2014), Shenzhen, China. ACM, December 2014
Abstract, BibTeX, Paper (.pdf)
Other files: Talk, Video
- Simon Mayer, Yassin N. Hassan, Gábor Sörös
Demo: Magic Lenses for Revealing Device Interactions in Smart Environments.
Live demo at the SIGGRAPH Asia 2014 Symposium on Mobile Graphics and Interactive Applications (MGIA 2014), Shenzhen, China. ACM, December 2014
Abstract, BibTeX, Paper (.pdf)
Other files: Video
- Jie Song, Gábor Sörös, Fabrizio Pece, Sean Fanello, Shahram Izadi, Cem Keskin, Otmar Hilliges
In-air Gestures Around Unmodified Mobile Devices.
Proceedings of the 27th ACM User Interface Software and Technology Symposium (UIST 2014), Honolulu, Hawaii. ACM, pp. 319-329, October 2014
Abstract, BibTeX, Paper (.pdf)
Other files: Talk, Video
- Jie Song, Gábor Sörös, Fabrizio Pece, Sean Fanello, Shahram Izadi, Cem Keskin, Otmar Hilliges
Demo: In-air Gestures Around Unmodified Mobile Devices.
Live demo and poster at the 13th European Conference on Computer Vision (ECCV 2014), Zurich, Switzerland. September 2014
Abstract, BibTeX
- Simon Mayer, Gábor Sörös
User Interface Beaming - Seamless Interaction with Smart Things using Personal Wearable Computers.
Proceedings of the 11th International Conference on Wearable and Implantable Body Sensor Networks (BSN 2014). pp. 46-49, Zurich, Switzerland, June 2014
Abstract, BibTeX, Paper (.pdf)
Other files: Talk
- Gábor Sörös
GPU-Accelerated Joint 1D and 2D Barcode Localization on Smartphones.
Proceedings of the 39th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2014), Florence, Italy. IEEE, pp. 5095-5099, May 2014
Abstract, BibTeX, Paper (.pdf)
Other files: Poster, Video
- Gábor Sörös, Christian Floerkemeier
Blur-Resistant Joint 1D and 2D Barcode Localization for Smartphones.
Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia (MUM 2013), Lulea, Sweden. ACM, December 2013
Abstract, BibTeX, Paper (.pdf)
Other files: Talk
- Gábor Sörös, Florian Daiber, Tomer Weller
Cyclo - A Personal Bike Coach Through the Glass.
Proceedings of SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications (MGIA 2013), Hong Kong, Hong Kong. ACM, November 2013
Abstract, BibTeX, Paper (.pdf)
Other files: Talk
- Simon Mayer, Markus Schalch, Marian George, Gábor Sörös
Poster: Device Recognition for Intuitive Interaction with the Web of Things.
Adjunct Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2013). Zurich, Switzerland, September 2013
Abstract, BibTeX, Paper (.pdf)
- Simon Mayer, Christian Beckel, Bram Scheidegger, Claude Barthels, Gábor Sörös
Demo: Uncovering Device Whispers in Smart Homes.
Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia (MUM 2012), Ulm, Germany. December 2012
Abstract, BibTeX, Paper (.pdf)
Other files: Poster
- Gábor Sörös, Christian Floerkemeier
Poster: Towards Next Generation Barcode Scanning.
Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia (MUM 2012), Ulm, Germany. December 2012
Abstract, BibTeX, Paper (.pdf)
Other files: Poster
- Gábor Sörös, Peter Rautek, Hartmut Seichter, Eduard Gröller
Augmented Visualization with Natural Feature Tracking.
Proceedings of the 10th ACM International Conference on Mobile and Ubiquitous Multimedia (MUM 2011), Beijing, China. pp. 4-12, December 2011
Abstract, BibTeX, Paper (.pdf)
Other files: Talk
- Robert Adelmann
An efficient bar code recognition engine for enabling mobile services.
PhD thesis No. 19721, ETH Zurich, Zurich, Switzerland, 2011
Abstract, BibTeX, Paper (.pdf)
- Felix von Reischach, Stephan Karpischek, Robert Adelmann, Florian Michahelles
An Evaluation of 1D Barcode Scanning on Mobile Phones.
Proceedings of Internet of Things 2010 International Conference (IoT 2010). Tokyo, Japan, November 2010
Abstract, BibTeX, Paper (.pdf)
- Iulia Ion, Marc Langheinrich, Ponnurangam Kumaraguru and Srdjan Capkun
Influence of User Perception, Security Needs, and Social Factors on Device Pairing Method Choices.
Proceedings of the Symposium on Usable Privacy and Security (SOUPS 2010). Redmond, WA, July 2010
Abstract, BibTeX, Paper (.pdf)
- Robert Adelmann, Marc Langheinrich
SPARK Rapid Prototyping Environment – Mobile Phone Development Made Easy.
Proceedings of Intelligent Interactive Assistance and Mobile Multimedia Computing (IMC09). Rostock, Germany, November 2009
BibTeX, Paper (.pdf)
- David H. Nguyen, Gabriela Marcu, Gillian R. Hayes, Khai N. Truong, James Scott, Marc Langheinrich, Christof Roduner
Encountering SenseCam: Personal Recording Technologies in Everyday Life.
Proceedings of the 11th International Conference on Ubiquitous Computing (UbiComp 2009), Orlando, FL, USA. ACM Press, pp. 165-174, New York, NY, USA, October 2009
Abstract, BibTeX, Paper (.pdf)
- Felix von Reischach, Florian Michahelles, Dominique Guinard, Robert Adelmann
An Evaluation of Product Identification Techniques for Mobile Phones.
Proceedings of Interact 2009 (IFIP TC13 Conference in Human-Computer Interaction). Uppsala, August 2009
Abstract, BibTeX, Paper (.pdf)
- Silvia Santini, Benedikt Ostermaier, Robert Adelmann
On the Use of Sensor Nodes and Mobile Phones for the Assessment of Noise Pollution Levels in Urban Environments.
Proceedings of the Sixth International Conference on Networked Sensing Systems (INSS 2009), Pittsburgh, PA, USA. June 2009
Abstract, BibTeX, Paper (.pdf)
- Dominique Guinard, Oliver Baecker, Florian Michahelles
Supporting a Mobile Lost and Found Community.
Proceedings of MobileHCI 2008 (ACM International Conference on Human-Computer Interaction with Mobile Devices and Services). Amsterdam, Netherlands, 2008, September 2008
BibTeX, Paper (.pdf)
- Robert Adelmann
BaToo - Enabling the Rapid Prototyping of Mobile Services to Retail Products.
Adjunct Proceedings of Internet of Things (IOT08). Zurich, Switzerland, 2008
BibTeX, Paper (.pdf)
- Robert Adelmann, Marc Langheinrich
Rapid Prototyping Platform for Mobile Phone Based Services on Retail Products.
Demo at the International Symposium on Ubiquitous Computing Systems (UCS 2007), Akihabara, Tokyo, Japan, November 2007
Abstract, BibTeX, Paper (.pdf)
- Philipp Bolliger, Moritz Köhler, Kay Römer
Facet: Towards a Smart Camera Network of Mobile Phones.
Proceedings of Autonomics 2007 (ACM First International Conference on Autonomic Computing and Communication Systems). Rome, Italy, October 2007
Abstract, BibTeX, Paper (.pdf)
- Philipp Bolliger, Benedikt Ostermaier
Koubachi: A Mobile Phone Widget to enable Affective Communication with Indoor Plants.
Adjunct Proceedings of MobileHCI, Mobile Interaction with the Real World Workshop (MIRW 2007). Singapore, September 2007
Abstract, BibTeX, Paper (.pdf)
- Christof Roduner, Marc Langheinrich, Christian Floerkemeier, Beat Schwarzentrub
Operating Appliances with Mobile Phones – Strengths and Limits of a Universal Interaction Device.
In: Anthony LaMarca, Marc Langheinrich, Khai Truong (Eds.): Pervasive Computing. 5th International Conference, PERVASIVE 2007, Toronto, Canada, May 13-16, 2007, Proceedings. LNCS, Vol. 4480, Springer, pp. 198-215, Berlin Heidelberg New York, May 2007
Abstract, BibTeX, Paper (.pdf)
- Christian Frank, Philipp Bolliger, Christof Roduner, Wolfgang Kellerer
Objects Calling Home: Locating Objects Using Mobile Phones.
Proceedings of the 5th International Conference on Pervasive Computing (Pervasive 2007). LNCS, Vol. 4480, Springer, Toronto, Canada, May 2007
Abstract, BibTeX, Paper (.pdf)
- Robert Adelmann
Mobile Phone Based Interaction with Everyday Products - On the Go.
International Conference on Next Generation Mobile Applications, Services and Technolgies (NGMAST). IEEE Computer Society, Cardiff, UK, 2007
BibTeX, Paper (.pdf)
Related Student Projects
The following table lists corresponding student projects in our group. Note that some descriptions will be in German.
Type | Title | Student | Supervisor | Semester |
L |
Facial Augmented Reality |
Andreas Hess | Gábor Sörös | HS 16 |
B |
Visible Light Communication for Smartglasses |
Pascal Josephy | Gábor Sörös, Stefan Schmid | FS 16 |
B |
Solfege Sign Recognition with a Wearable Camera |
Julia Giger | Gábor Sörös | FS 16 |
M |
Online Gyroscope-Camera Autocalibration for Image Enhancement on Smartphones |
Luc Humair | Gábor Sörös | FS 15 |
L |
Image Processing on Mobile GPU |
Carlo Beltrame | Gábor Sörös | FS 15 |
B |
Image Processing on Mobile GPU |
Marc Fischer | Gábor Sörös | FS 15 |
B |
Wearable Shopping Assistant |
Mauro Guerini | Gábor Sörös | FS 15 |
M |
Blur Removal from Document Images |
Stephan Semmler | Gábor Sörös | HS 14 |
B |
Multiframe Blur Estimation using Inertial Measurement Sensors |
Severin Münger | Gábor Sörös | FS 14 |
B |
Smart Playing Cards with Smart Glasses |
Michael Franz | Gábor Sörös | FS 14 |
B |
Developing a Multi-Level Visual Code |
David Chettrit | Gábor Sörös, Leyna Sadamori | FS 14 |
B |
Rapid Object Reconstruction for Product Augmented Reality |
Sandro Lombardi | Gábor Sörös | FS 14 |
B |
A Biking Assistant on a Wearable Computer |
Andy Zimmermann | Gábor Sörös | HS 13 |
B |
Optical Character Recognition with a Head-Mounted Camera |
Thomas Knell | Gábor Sörös | FS 13 |
B |
A Magic Lens for Visualizing Device Interactions |
Markus Schalch | Simon Mayer, Gábor Sörös, Marian George | FS 13 |
B |
Motion Blur Compensation using Inertial Measurement Sensors |
Carlo Beltrame | Gábor Sörös | FS 13 |
M |
Optical Character Recognition on Mobile Phones (in cooperation with MIT) |
Ekansh Anand | Christian Floerkemeier, Gábor Sörös | FS 12 |
L |
Uncovering Device Whispers |
Bram Scheidegger, Claude Barthels | Simon Mayer, Christian Beckel, Gábor Sörös | FS 12 |
B |
Creating Briefings from Heterogeneous Sources (II) |
Marco Poltera | Simon Mayer | FS 12 |
B |
Verifying Cloud Encryption Credentials through Mobile Device Interactions |
Marius Fehr | Iulia Ion | FS 12 |
B |
Optical Character Recognition on Mobile Phones |
Luc Humair | Christian Floerkemeier, Gábor Sörös | FS 12 |
B |
Creating Briefings from Heterogeneous Sources (I) |
Raffael Buff | Simon Mayer | FS 12 |
B |
Adaptive User Interfaces for Smart Things |
Andreas Tschofen | Simon Mayer | FS 11 |
M |
Crowdsourcing location information to improve indoor localization |
Luba Rogoleva | Philipp Bolliger | HS 09 |
S |
Entwicklung einer Sensor-Anwendung für Mobiltelefone (Sensing with Mobile Phones) |
Adrian Helfenstein | Benedikt Ostermaier | FS 09 |
S |
Prototyping a RESTPhone |
Moritz Hartmeier | Benedikt Ostermaier | FS 09 |
L |
Koubachi - iPhone Client |
Mike Godenzi, Gilad Geron | Philipp Bolliger, Moritz Köhler | FS 09 |
B |
Redpin - Next Generation Server |
Pascal Brogle | Philipp Bolliger | FS 09 |
B |
Redpin - iPhone Client |
Pascal Brogle | Philipp Bolliger | FS 09 |
S |
Test-Robot for Mobile Phone Cameras |
Dessovic Raoul | Robert Adelmann | HS 08 |
M |
Integration of Mobile Phones into Smart Environments for Education and Entertainment |
Roger Seidel | Steve Hinske | FS 08 |
M |
Smart Environment Visualization and Interaction |
Alex Suciu | Robert Adelmann | SS 07 |
L |
Smart Notifier |
Vlatko Davidovski | Benedikt Ostermaier | SS 07 |
S |
"Mobile Phone Blogger", oder was sieht mein Natel? |
Angelo Rosenfelder | Moritz Köhler | HS 07 |
S |
Spatial Selection Using Mobile Phones |
Breu Urs | Robert Adelmann | SS 06 |
S |
Information Gathering |
Ziroli Plutschow | Robert Adelmann | SS 06 |
L |
WorldWindow Augmented Reality System |
Steven Fluck, Jonas Haller | Robert Adelmann | SS 06 |
S |
Mobile Barcode |
Bettina Polasek | Robert Adelmann | WS 05/06 |
|