SIXTH SENSE TECHNOLOGY
Basic Aim:
To create a Sixth Sense device which works of the
principles of gesture recognition and image processing to capture, zoom(in and
out) and toggle pictures with ease just by the help of colored caps worn on the
fingertips of the user.
Motivation:
When we were thinking of projects for
the summers to execute, we thought of making a hand, which would do exactly
what our hand is which could have been something brilliant in robotics.
Then
we came across a TED talk by Pranav Mistry where he was talking about how we
can connect the physical world with the digital world using Sixth Sense. It
looked simply amazing
how
he was able to just show his fingers in front of his camera (phone) and the
camera would click pictures.
This just enthralled us and was just something we
had to make considering it’s sheer brillianc
Theory:
‘SixthSense’
is a wearable gestural physical world around us with digital information and
lets us
use natural hand
gestures to interact with that information.
Every
one of us is aware of the five basic senses –seeing, feeling, smelling, tasting
and hearing. These senses have evolved through millions of years. Whenever we
encounter a new object/experience our natural senses tries to analyse that
experience and the information that is obtained is used to modify our
interaction with the environment. But in this new age of technologies the most
important information that helps one to make right decision is something that
cannot be perceived and analysed by our natural senses. That information is the
data in the digital form, and it is available to everyone through sources like
internet. The sixth sense technology concept is an effort to connect this data
in the digital world in to the real world.
Although
miniaturized versions of computers help us to connect
to
the digital world even while device as of now which gives a direct link between
the digital
world and our
physical interaction with the real world. Usually
the information’sionallyon aarepapeor storedadigital
tra storage device. Sixth sense technology helps to bridge this gap
between tangible
and non-tangible world.
The
sixth sense technology was developed by Pranav Mistry, a PhD student in the
Fluid Interfaces Group at the MIT Media Lab. According to him the sixth sense
technology has a Web 4.0 view of human and machine interactions.
The SixthSense technology contains a
pocket projector, a mirror and a camera contained in a pendant-like, wearable
device. Both the projector the camera and sensors are connected to
a coding device (in our case-a laptop) in the user’s
po projector projects visual information enabling surfaces, walls
and
physical objects around us to be used as interfaces; while the camera
recognizes and tracks users' hand gestures and physical objects using
computer-vision based techniques. The software program processes the video
stream data captured by the camera and tracks the locations of the colored
markers
(visual
tracking fiducials) at the tips of the user’s fingers movements and
arrangements of these fiducials are interpreted
into
gestures that act as interaction instructions for the projected application
interfaces. The Sixth Sense prototype is used to implement several applications
that have shown the usefulness, viability and flexibility of the system.
The Sixth Sense:
SixthSense is a wearable gestural
interface that augments the physical world around us with digital information
and lets us use natural hand gestures to interact with that information.
Camera
It captures the image of the obj hand gesture. The
camera recognizes individuals, images,
pictures,
gestures that user makes with his hand. The camera then sends this data to a
smart phone for processing. Basically the camera forms a digital eye, which
connects to the world of digital information.
Colored Marker
There
are color markers placed at the tip of users finger.
Marking
the user’sred,yellowgreenfingersandblue with colored tape helps the webcam to
recognize the hand gestures.
The
movements and arrangement of these markers are interpreted into gestures that
act as a interaction instruction for the projected application interfaces.
Microsoft-enabled
Laptop
The
SixthSense device consists of a laptop which process the data send by the
camera. The laptop interprets the hand gestures with help of the colored
markers placed at the fingertips.
Dev-C++
was the programmer used for all codes made. MinGW was the compiler used. OpenCV
libraries were used which contained all functions sufficient to make our
project.
Projector
The
information that is interpreted through the smart phone can be projected into
any surface. The projector projects the visual information enabling surfaces
and physical objects to be used as interfaces. The projector itself consists of
a battery which have 3 hours of battery life .A tiny LED projector displays the
data sent from the smart phone on any surface in view- object, wall or person.
The Original Device:
Technologies that are related to Sixth Sense
Devices
Augmented
Reality
The augmented Reality is a visualization technology
that allows the user to experience the virtual experience added over real world
in real time. With the help of advanced AR technology the information about the
surrounding real world of the user becomes interactive and digitally usable.
Artificial information about the environment and the objects in it can be
stored and retrieved as an information layer on top of the real world view.
When we compare the spectrum between virtual reality, which creates immersive,
computer-generated environments, and the
real
world, augmented reality is closer to the real world. Augmented reality adds
graphics, sounds, haptic feedback and smell to the natural world as it exists.
Both video games and cell phones are driving the development of augmented
reality. The augmented systems will also
superimpose graphics for every perspective available and try adjusting to every
movement of the user's head and eyes. The three basic components of an
augmented reality system are the head-mounted display, tracking system and
mobile computer for the hardware. The main goal of this new technology is to
merge these three components into a highly portable unit much like a
combination of a high tech Walkman and an ordinary pair or eyeglasses. The
head-mounted display used in augmented reality systems will enable the user to
view superimposed graphics and text created by the system. Another component of
an augmented reality system is its tracking and orientation system. This system
pinpoints the user's location in reference to his surroundings and additionally
tracks the user's eye and head movements. Augmented reality systems will need
highly mobile computers.
As
of now many computers aren’t option.
Gesture
Recognition
It
is a technology which is aimed at interpreting human gestures with the help of
mathematical algorithms. Gesture recognition technique basically focuses on the
emotion recognition from the face and hand gesture recognition. Gender
recognition technique enables humans to interact with computers in a more
direct way without using any external interfacing devices. It can provide a
much better alternative to text user interfaces and graphical user interface
which requires the need of a keyboard or mouse to interact with the computer.
Interfaces which solely depends on the gestures requires precise hand pose
tracking. In the early versions of gesture recognition process special type of
hand gloves which provide information about hand position
orientation and flux of the fingers. In the
SixthSense devices colored bands are used for this purpose. Once hand pose has
been captured the gestures can be recognised using different techniques. Neural
network approaches or statistical templates are the commonly used techniques
used for the recognition purposes. This technique has a high accuracy usually
showing accuracy of more than 95%. Time dependent neural network will also be
used for real time recognition of the gestures.
Computer Vision
Computer
vision is the technology in which machines are able to interpret/extract
necessary information from an image. Computer vision technology includes
various fields like image processing, image analysis and machine vision. It
includes certain aspect of artificial intelligence techniques like pattern
recognition. The machines which implement computer vision techniques require
image sensors which detect electromagnetic radiation which are usually in the
form of ultraviolet rays or light rays. The computer vision find itself
applicable in varies field of interest. One such field is bio medical image
processing.
It’s also used in autonomousThecomputer veh vision
technique basically includes four processes.
1. Recognition:
One of the main task of computer vision technique is to determine whether the
particular object contain the useful data or not.
2. Motion
Analysis: Motion analysis includes several tasks related to estimation of
motion where an image sequence is processed continuously to detect the velocity
at each point of the image or in the 3D scene.
3. Scene
Reconstruction: Computer vision technique employs several methods to recreate a
3D image from the available images of a scene.
4. Image
Restoration: The main of aim of this step is to remove noise from an given
image. The simplest method involves using simple filters like low pass or median
filters. In order to get better quality images more complex methods like
Inpainting are used.
Portable
One
of the main advantages of the sixth sense devices is its small size and
portability. It can be easily carried around without any difficulty. The
prototype of the sixth sense is deigned in such a way that it gives more
importance to the portability factor. All the devices are light in weight and
the smart phone can easily fit into the users pocket.
Cost effective
The
cost incurred for the construction of the sixth sense proto type is quiet low.
It was made from parts collected together from common devices. And a typical
sixth sense device cost up to $300. The sixth sense devices have not been made
in large scale
for commercial purpose. Once that happens it’s
almost c that the device will cost much lower than the current price.
Connectedness
between real world and digital world
Forming
a connection between the real world and the digital world was the main aim of
the sixth sense technology.
Data access
directly from the machines in real time
With
help of a sixth sense device the user can easily access data
from any machine at real time sp any machine-human
interface to access the data. The data access
through
recognition of hand gestures is much easier and user friendlier compared to the
text user interface or graphical user interface which requires keyboard or
mouse.
Open source
software
The
software that is used to interpret and analysis the data collected by the
device is made open source. This enables other developers to contribute to the
development of the system.
APPLICATION
The
sixth sense technology finds a lot of application in the modern world. The sixth
sense devices bridge the gap by bringing the digital world into the real world
and in that process allowing the users to interact with the information without
the help of any machine interfaces. Prototypes of the sixth sense device have
demonstrated viability, usefulness and flexibility of this new technology.
According to the words of its developers extend the of use of this new device
is only limited by the imagination of human beings. Some practical applications
of the sixth sense technology is given below
Map Application
With the help of a map application the user can call
upon any map of his/her choice and navigate through them by projecting the map
on to any surface .By using the thumb and index fingers movements the user can
zoom in, zoom out or pan the selected map.
Taking Pictures
Another
application of sixth sense devices is the implementation of a gestural camera.
This camera takes the photo of the location user is looking at by detecting the
framing gesture. After taking the desired number of photos we can project them
onto any surfaces and then use gestures to sort through those photos and
organize and resize them.
The user taking making a pose gesture.
The camera takes the photo of the location
Drawing
Application
The
drawing application allows the user you to draw on any
surface by tracking the fingertip movements of th
finger. The pictures that are drawn by the user can be stored and
replaced
on any other surface. The user can also shuffle through various pictures and
drawing by using the hand gesture movements
Making Calls
We can make calls with help of sixth
sense device .The sixth sense device is used to project the keypad into your
palm and the using that virtual keypad we can make calls to anyone.
Interacting with
physical objects
The
SixthSense system also helps to interact with physical objects we use in a
better way. It augments physical objects by projecting more information about
these objects projected on
them.
For example, a gesture of wrist projects an analog watch on the users hand.
Similarly a
newspaper
can show live video news or dynamic information can be provided on a regular
piece of paper
Newspaper
showing live feeds an analog
watch
projected into the wrist
Getting
Information
Sixth
sense devices can be used for getting various information relating to our
everyday life by getting in contact with objects
Product
Information:
Sixth
sense technology uses marker technology or image recognition techniques to
recognize the objects we pick in our hand and then provide information relating
to that product.
Book
Information:
By
holding and shuffling through the book pages, the sixth sense provides Amazon
ratings on that book, other reviews and other relevant things related to the
book.
Flight Updates:
With
the help of the sixth sense technology it is no longer required to log into any
sites for checking the status of the flights. The system will recognize your
boarding pass and let you know whether the flight is on time or not.
Information
About People:
With help of face recognition techniques the sixth
sense devices are able to provide information about the people when we meet
them. The sensor detects the face and checks the data base for the relevant
information. The system will then project the relevant information about a
person like what they do, where they work.
0 comments:
Post a Comment