Creating your First Multi-touch Table

 
It has been a long time since I've contributed to this website--going on to a year almost--I was trying to accomplish a personal goal that was in the making since my early high school years. This instructable has taken 2 years of independent research and setbacks, but it is finally here. I present to you the complete guide to building a personal Multi-touch table at home with basic design and engineering skills ( I will attempt to fully explain every detail so that even a beginner can find success!). Most mathematics have been performed so the only fear regarding this project is the fact that it will take some time to put together . . . and some money.

As with every project, here is a bit of back-story:

When I was in my freshman year of high school, I was invited to attend an engineering "field day" taking place in Alabama's Redstone Arsenal. Although most of what we were shown during this trip were projects that I am not able to speak about, one of the more relaxed projects was the Multi-touch table. I am not allowed to say what the table was being used for--I've forgotten either way--but I can speak about its features. The table shown to us was a 30" Microsoft Pixelsense (then known as the Surface, before the tablet took its name) and to me, it was glorious. As a connoisseur of Natural User Interfaces (NUI), I'd always been fascinated by 3D spatial projections ( what most people call holograms) and anything of the like. I had never seen of these before so it would be easy for anyone to imagine my excitement. I asked the Project manager about what it would take to work with one of these and how I could be involved. You could imagine my disappointment when he told me that although it was simple, it wasn't anything that I could ever make by myself. This very conversation sparked a 5 year personal vendetta against that one employee that led to the creation of this instructable--as a hobbyist. I enrolled in optic courses and searched through every crevice of the Web until I had enough information to build my own table. I had two very important personal objectives:

1) Educational: To gain experience in NUI technologies and help spread this knowledge to students willing to learn via independent projects. I planned to stimulate interest with the introduction of a hobbyist-made Multi-touch table in order to engage myself deeper into NUI technologies
2) To create a product that was better than the table I was shown in my early high school years: bigger, faster, cheaper, and more efficient.

Thus, I can now say that the goal has been met. Now I can share this knowledge with all of you.

Also, a great amount of credit shall go to these mentioned organizations as they facilitated my goal. Without them, this would have taken much longer!

• NUI group--this collective of hobbyists and professionals dealing with NUI technologies shall gain the bulk of my credit. They taught me everything I needed to theoretically build my project and assisted me through every setback I encountered. They are also responsible for teaching me--above all things--how to conduct my own research. I strongly advise anyone who reads this to visit their website and forums.
• Ideum-- who would have thought that an established company would be so eager to lend a hand to an enthusiastic student? I literally called the hardware specialist at the time and asked him a bunch of questions, and he replied to each with informative tips. This company also led me to discover the NUI group and pushed me toward making my project bigger. If you get a chance, check them out too. If anything, they are a great source for inspiration.

• This project was conducted with 3 other friends, and the table is currently housed in one of the member's college dorms. As I am writing this alone and since I conducted all of the planning and designing, I will refer to myself when I speak of things that are design related. I may also refer to the group as a collective in the article, hence my using of "we" on the occasion.
• This will be a very long instructable. My aim is to make the most detailed instructable in order to increase interest in NUI tech, but above all, for you to understand all that you are doing as you are doing it.
• You may PM me for clarifications if needed.
• I may use the word "warez" if referencing to the collective software as a plural noun.
• Each section is pre-labeled in the title, allowing for your easy navigation. "Information" refers to my explaining theories, plans, mathematical or physical concepts, tools, etc. "Hardware" refers to sections where I outline the steps to take in actually building the hardware for the table. Finally, "software" or "warez" refers to sections where I explain how to set up software.

List of Contributors: As you may have noticed, this account is the property of a collective of currently enrolled college students. The list of contributors for this project are:

Step 1: The Planning (Information)

We are going to be building a 47" Planar Infrared-based Multi-touch table using the Diffused Surface Illumination (DSI) Technique. This technique is not recommended if you aim to read objects (that are not fingers)--also known as fiducials--unless you plan on not using the diffusive panel of the LCD. It is also not the best to implement when using an LCD, but it is the most convenient due to requiring less materials and work. I chose this one because of the convenience.

Theory:
By Diffused Surface Illumination (DSI), we place Infrared (IR) Light-Emitting Diodes (LEDs) around the edges of a special type of acrylic known as Endlighten Acrylic. This acrylic is manufactured with special properties akin to having microscopic mirrors that evenly distribute the light across the surface of the acrylic. When your finger touches the surface, it creates a "blob" on the surface--it interrupts the evenness of the light. This blob is then to be read by the specialized camera and recorded as a "touch event".


Comparison to PixelSense:

The first PixelSense used a different technique called Rear-Diffused Illumination ( Rear DI). By Rear DI, the maker is required to use IR illuminators on the underside of the screen. Because of this, it is nearly impossible to create using LCDs due to the IR light reflecting from the back panel. Thus, people will usually use a short throw projector (usually more expensive if you want HD quality). All in all, DSI allows for higher visual quality at a reduced price. Input detection remains the same.

Why make one of these when you have a smartphone?

Typically, smartphones and tablets use either capacitive, resistive, or surface acoustic wave screens in order to detect user touch points. Capacitive uses the electrical properties of human bodies in order to detect user inputs at certain points (the skin of your fingertips). Thus, devices using this screen are highly sensitive and neglect pressure--meaning that users cannot use regular styluses or gloved hands. In addition to this, the cost for capacitive layering on a large scale project would be tremendous.

Resistive screens simply rely on pressure as they are built by layering screens. For resistive, there is an inconvenience in that building this setup would require the user to create layering patterns with little promise of multi-input detection.

In Surface Acoustic Wave (SAW) screens, when a panel is touched, the ultrasonic waves exerted in the panels are absorbed at those points and this registers a position for touch events to the controller for processing. This would be burdensome if applied to larger screens.

However, infrared technology holds the advantage of being able to deliver superior images without an overlay while granting the ability to detect most objects. Also, the ability to use fiducial markers (programmable blocks). Also, they tend to have the higher number of Maximum touch inputs.

What is the first thing to worry about after I have my design plan ready?

The first thing you should worry about is your LCD TV. You won't be able to use just any TV as there may be issues. I will explain this in more detail soon.

Step 2: LCDs, the FFC issue, and Before Ripping (Information)

The LCD is perhaps the most annoying hardware component regarding the setup of an LCD DSI table due to the FFC Issue--I will get to this soon. But first, a bit of insight: It is mildly important to understand how LCDs work for the purpose of explaining how the table works. Also, since we are taking one apart, it is important that we understand what each component does.

I. LCD theory
LCDs are an amazing piece of technology that remind us of how amazing and progressed our technological prowess has come. At the core, they rely on Liquid Crystals (LC). LCs are materials that are literally in a state between liquid and solid states of matter--this means that their particles maintain their orientation, yet they are able to move around to different positions akin to liquid state materials. Thus, they are highly sensitive to temperature changes ( they must be at an exact temperature to maintain their odd state). For LCD screens, LC particles are usually arranged in a twisted nematic phase. This only means that they are affected by electric current (nematic) and are naturally twisted. When we apply current to the LC substrate, the LCs untwist varying degrees depending on the voltage--which allows them to react predictably in controlling the passage of light. The orientation of the particles in the natural state depend on either a magnetic field or microscopic grooves. The change in orientation (for most LCDs) is described as either smetic c or chiral nematic. The smetic c orientation holds that the particles are layered in such a way that they tilt slightly in angles with each layer while the chiral orientation sees the molecules untwist slightly from layer to layer.

II. LCD creation
For the LCD screen construction there are 4 factors that are fundamental:

• Light can be Polarized
• LCs can transmit and change polarized light
• LC structure changes with current
• There are transparent substances that conduct electricity

There are many types of LCD screens such as Passive and Active Matrix LCDs, but since we are most likely only dealing with an Active Matrix, I will only explain this one. Active Matrix LCDs depend on Thin Film Transistors (TFT) which are essentially tiny switching transistors and capacitors. They are arranged in a matrix fashion on a glass substrate and to turn a pixel on, the row corresponding to that transistor is turned on and a charge is sent through that column. Since the other rows are off, only the capacitor at that row is receiving a charge--theoretically knowing that Capacitors can change current instantaneously, this is explained and we also factor that the charge can be held until the next refresh cycle. Controlling the voltage, one can untwist the LC enough to moderate the amount of light that passes. This allows for the creation of a gray-scale.

So how do we get color?
Each Pixel has 3 sub-pixels (red-blue-green{RGB}) and they each have their own transistor. each sub-pixel allows for 256 shades within their own gray-scale for a grand total of 16.8 million possible colors on your screen. this is true unless you have a fancy yellow pixel LCD display as we get the possible colors from 255^3 for the 3 sub-pixels.

How many transistors are we talking about?

Depends on the display. A 1024x768 TV has 1024 columns and 768 rows, so if we multiply that by 3 pixels, we get 2,359,296 TFTs etched onto the glass panel. If that doesn't impress you, wait until you see how thin the screen really is and what happens when you turn it on away from its casing!

Why should you care about all of this?
In building your table, you are going to have to make predictions and decisions. You won't always have a perfect display built. Sometimes, you're going to have to figure things out from the fundamentals in order to make sure you didn't just waste $$$$. Also, depending on your setup, you may have to predict the display size of your LCD based on the properties of the screen. More importantly, this is knowledge! so take it in!

III. Things I should worry about in regards to the TV I will use for the table:

1. The FFC Issue as defined on the NUI Group Forums is when the FFC cables are either too small or fragile to be manipulated into building a Multi touch setup. The Flat Flex Cable is the cable that you will find connecting the LCD matrix to the Control boards. This can become a problem as they are hard to replace and if the get in the way of the construction, they may cause accidental damage. Fortunately, there are links to databases with TVs that do not have this problem and are perfect for Multi touch setups:
2. The Aspect Ratio: You need to know your aspect ratio so that you can approximate the size of your LCD in order to create your chassis, or box. Most LCDs nowadays are 16:9 since this is the HDTV Standard. That is, 9 units of height for 16 units of width. Use similar triangles (yes, math) in order to determine the size of your LCD screen. Of course, this is only necessary in the case you do not want to rip the TV apart until you have everything else (Acrylic, wood, etc).
3. Contrast Ratio: This is the measure between the brightest possible image and the darkest possible image. White/Black = Contrast Ratio. Unfortunately, there is no actual metric to standardize this so companies just exaggerate things such as 1000000000:1 (who knows what that means?) This matters only in terms of vanity--say you wanted the clearest possible image to be displayed on your LCD. An LCD with a high dynamic ratio is better than an LCD with a native that gives the same ratio; however, an LCD with a higher native ratio is superior. Native describes what the LCD is able to produce on its own and dynamic refers to using circuitry to assist the panel in accordance to ambient light.

What about glare when the LCD is laying down?
The LCD screen comes with a set viewing angle that ensures the best possible picture--most consumer don't know this and miss out on the full experience. A Bias angle is designed to offset this angle. Because of this, your angle of acceptable viewing is made much larger. There is still an area of unacceptable contrast, but you will most likely not reach it.

Step 3: Steps in determining your LCD screen dimensions before-hand (optional math information)

1. The first thing to do in determining the LCD size is to find out what your aspect ratio is. The beautiful thing is that LCD retailers are required to release a specs sheet concerning the capabilities and basic information regarding the TV. In these, the most notable of specs is the aspect ratio--usually 5:4 or 16:9.
2. Next, determine the size of your diagonal from these sheets. A display that is 47" isn't exactly 47" (it may actually be 46.85 or 46.96, which actually matters). Find this on your spec sheets and write it down.
3. Next, use triangle-angle relationships to determine the angle opposite the horizontal and vertical measurement of your display using the aspect ratio.
   1. For example, a 16:9 ratio LCD will have 16 units of measure for the horizontal component and 9 for the vertical. The angle opposite is the given by cos (16/proper diagonal measure). For a television of a 46.96" diagonal, Cos(16/46.96") = 60.64224646 degrees, which makes sense.
4. Next, setup a similar triangle diagram where the 16:9 is within the LCD. You are to find the X and Y components of the outer triangle. You use law of sines with the angle determined above to find this.
   1. For example, following our calculations, [Sin(60.642)/9] = [(x/46.96")/x] given that y = 9x/16.
   2. This will give you an approximation of 40.92918722/23.02266781 which falls under 16:9 ratio when reduced. This however, is not exact but fairly close.

Step 4: Making a Box . . . ! (Hardware)

The First step of the hardware process is to build a box that will hold all of your components together. This step will be where your design starts taking a new life depending on whether you want a coffee table, a display monitor, a station, etc. I am currently in the process of modifying my box and make a coffee table. Once that is completed, if anyone is interested, I can upload some pictures and describe the build process. The pictures above are very simplistic due to our table being built as a prototype that was ultimately going to be modified either way.

I. Things to take into consideration when building the box:

1. Is your box the right size to hold your LCD screen? This is where aspect ratio comes into play. You need to make sure your layers won't fall through and that the LCD cables/components won't get in the way. Also, make sure you can make it mobile (it can fit through doors)
2. Where will your computer be? You may not want to put the computer or the LCD components inside the box unless you are willing to include compartments on the underside for them as they will interrupt the image on the LCD and the camera's ability to track your fingers.
3. Ventilation: The Cameras won't get too hot, but if you do decide to conceal the computer and LCD components (which does make it look nicer), you may want to invest on some ventilation. You can use a couple of in/outtake fans for this if you want.
4. Are you accommodating for cabling? Make sure you have a plan in mind for neatly placing the cables in your table. One cable over the camera makes a significant amount of interference.
5. Is your structure strong enough to hold the Screen layers? you have about 32-45 lbs of screen to hold and the additional weight due to people putting their hands on your table.
6. A door? It may be a convenience to add a door so that you work with the LEDs and Cameras without having to remove your layers.
7. Is your box closed? You should enclose your setup so that the white LED light do not escape your box
8. Remember that Home Depot will cut the wood for you: You can get the exact amount you need and pay less. You must do this before checkout though.

II. Things we should discuss:

1. Your LCD panel has a back plate that acts as a partially reflective mirror in a sense. On your TV, you will mostly likely find fluorescent tubes. These are known as the back-light and their sole purpose is to provide the base light that the LCD screen will use. The panel also contains the polarizers necessary for the LCs to control the light. We are going to replace this entirely with the LEDs. Why? The tubes are too weak to provide the lighting necessary for the LCD to work once the distance is increased--and you must increase the distance so that you may accommodate for your cameras. Also, the tubes are very fragile and risky. The white LEDs will be placed on the bottom of the box in order to make the screen show an image. Additionally, for this to work, they must be place around the cameras so that they are not in the way. the cameras should also be elevated so that you don't get too much light interference.
2. You may want to first sketch your design or create a model using a CAD program. You can get Google Sketch-up and it will work wonders for free. It also has a built-in tutorial and gallery. You can find the Ideum MT50 table CAD file there too if you want a bit of inspiration
3. You MUST calculate the distance needed between your camera and screen in order to determine the size of your box. I used two cameras so that I could stitch the two feeds together and reduce my height by three times. Normally, this means having to calculate angles and ray diagrams. Thankfully, the NUI forums has a thread where Peau Productions created a lens calculator that may help simplify this. Please use this before purchasing your cameras or buying the wood. Also, consider looking through the projects on this forum.
4. Consider putting the table on wheels. It will increase mobility and give your design an edge.

Step 5: Ventilation concepts and laying the support (Hardware)

As laid out in the previous step, you may want to start thinking about ventilation for your chassis. As pictured above, our simple setup opted for neglecting this step as it was only a prototype. You may want to consider it depending on your component and screen positioning. Remember that IR is also heat, so you may need a system to remedy problems relating to this if they arise. As far as layering for now, consider adding various lips and edges to your table. Remember that you will need a place to house a mouse and keyboard for initial setup and troubleshooting. Also, If you are considering to lay a shell over your box as a design choice (as we did), you'll need a way to support the acrylic. We personally ordered our Acrylic to be larger than our LCD screen in order to have the effect of a floating LCD screen. If you aren't expecting any children or aggressive people to touch your screen--the Endligthen layer is very easily scratched--you may even replace the extruded FF with the abrasion resistant layer. All in all, lay a piece of acrylic for support. If you are a careful person, use the abrasion resistant layer as a support. If you don't want to risk anything going wrong, use the extruded FF.

Step 6: Re-purposing your Television--a step by step guide (Hardware)

The following is a guide to ripping your TV apart. Remember, proceed only after you have either:

• a. Made sure your TV avoids the FFC issue
• b. Are comfortable enough to rip your TV apart.
• c. Understand that I am not to be held responsible for the damage you cause, nor the risks to your lives.
• d. Have read this section at least once.

Taking apart the TV:

1. Of course, the first thing to do is turn off the power and remove the AC cable and gently lay the television screen-down on a flat and stable surface.
2. With a screwdriver, unscrew the screws holding the back-plate in place and remove the back-plate. You should see something similar to the first picture above. Luckily, the Vizio I selected is very simple and doesn't have too many boards. In the case you didn't stick with the Vizio and opted for something else, here goes a breakdown of what you'll see:
   • The Main Board: This is the most obvious piece you'll find. It is a circuit board that contains the audio and video inputs for the TV. You'll see the HDMI, USB, and S-video ports. This houses the CPU (on some TVs) and is also referred to as the Motherboard or the Main Module.
   • Power Board: This is the odd looking circuit board. 98% of the time, it will be a brownish-yellow color. This one is the bulkiest/heaviest of the boards and houses a couple of capacitors, inductors, and heat-sinks. This is most obviously the power supply board as this is where the AC cord will connect to. Fun fact: Many times, people will replace a TV they believe is damaged when the only actual problem may be that a single Capacitor has blown. Seeing as how one of these may cost a couple of cents, it's funny how we'd pay over $400 for a $1 fix.
   • Inverter Board: Long thin board along the edge of the display. Contains several transformers and performs logic operations regarding back-light and the display.
   • T-Con Board: the Smallest board and is a Logic Board. Mostly contains the CPU (on some TVs) and processing chips.
   • Speakers: you can either keep them or replace them with better ones. Choice is yours.
   • LED for TV symbol: a simple LED that lights up as an indicator that the TV is on. You may want to keep this around.
   • Circuit board for buttons: This is just a circuit board found on the side where the on/off, channel, volume, etc switches are. You may not need this.
   • A set of long flat cables: these are the FFCs. The Vizio has 3 ft cables which is beyond perfect.
3. Take a picture of everything. You have to remember each connection for each component so that it can reconnect.
4. Safely remove all the components and unscrew the boards from the back. Remove the speakers and everything. The connectors are usually Zero Insertion Force (ZIF) connectors. Keep one end of both of the FFC cables connected to the Inverter Board.
5. You will see a metal back-plate with holes. Through those holes, you will likely see white sheets. Those are the diffusers and we are trying to get to those. Unscrew the screws holding the plate together and remove the plate. To do this, we have to take the front plastic casing off so you'll have to flip the TV. Do this carefully as you do not want to risk damaging the LCD panel.
6. Take the casing off and work your way unscrewing until you get to the LCD panel. Gently and safely remove the connectors to the panel. The green and brown connectors DO NOT come off. If it looks like it won't come off, it most likely won't. Be very careful here.
7. Lay the LCD panel safely on a flat surface and wrap it with a blanket or something that will protect it. The panel is ~2 mm thick and thus, easily warped. Have a friend help you handle it so you distribute the weight evenly. Now think back to all that theory you read in the previous section and be amazed at how compact the panel is.
8. Right under the LCD are the white diffusers. Take care as they are likely to be statically charged. The diffusers serve to evenly distribute the white light under the panel from the back-light. You may not need all three though. We opted to use the thickest one as the others did nothing for us. The case may be different for you. For now, just keep them until the testing of the LCD display.
9. Do what you will with the back-light. If you opt for a size < 25" diagonal, you may actually make the factory light work and circumvent the cost of white LEDs. If not, do what you will with it--for the purpose of this table, it is no longer needed.

Step 7: Giving Life to your table (Hardware)

Recall the last section where I advised you to take pictures of each and every connection. This is where all will come in handy. In this section, we will be essentially rebuilding the LCD TV on your box. How you do this is going to depend solely on how you chose to design your box a couple of sections back. As for our team, we decided to go with designing an inner box that would later be concealed within another outer box. We simply used wood screws to screw the boards onto the back side of the inner box. The important thing to consider is to be careful that you don't crack the boards while screwing them onto the wood. For now, disregard the LCD panel. Do not place it until the next section as we must discuss the layering for the table. However, you do have the option of placing your computer along with the boards of the LCD. There is a great amount of flexibility when choosing a computer. To be honest, as long as it has over 512 MB RAM and is HDMI compatible, the computer will perform nicely. Any recent/decent PC will do--even a laptop if you so wish. We'll talk operating systems later on. For now, just choose your favorite as we have to set everything up.

Step 8: LED Setup (Hardware/Circuits)

Now we will set up the LED back-light and Cameras. This part is fairly simple and will only require minimal knowledge of soldering.

I. Camera setup

1. Assuming you calculated the height of your table based on your camera lens as I advised in the previous planning sections (or have prepared an appropriate platform for elevating the cameras to their needed height), place the cameras where they need to be. You need to perform the above calculations for centering your cameras as well.
   1. ****If you do not yet understand how to perform any of these equations, you can PM me and I will try to help. You simply use the FOV for the camera to determine a distance from the screen where the cameras can see portion desired from the screen. This should give you an idea of proper height and placement for the cameras.
2. As pictured above, we used Duct Tape to keep the cameras steady. This is because we were still experimenting and making sure that our calculations were correct. The cameras I referenced you to from Peau Productions have certain sections for you to screw the in place when you are satisfied with their position. I should advise you not to do that until you have first configured all the software needed.
3. Next, run the USB extensions to your computer. Make sure you keep the cables neatly pinned to the surface of the bottom so as to not have them interfere with the camera lens or the lights.

II. LED setup

1. We are concerned with the white LEDs here. Make sure you have the white LEDs on hand and not the IR.
2. The reel is designed in a manner that you can cut the reel at certain points to make smaller sections. You can tell where these sections are because you can see a scissors symbol. Also, the symbols for positive (+) and GND (-) will meet their counterparts here.
3. Determine your section size, cut, and paste to the bottom around the cameras as pictured above (the underside of the LEDs is a 3M adhesive).
4. Solder the positive to the positive and the negative to the negative as trivial. The power adapter for the LEDs also needs to be connected to these in some way. You may solder it directly, buy a female jack, or just twist and tape.

Step 9: Acrylic Setup/ IR setup (Hardware)

Find your Endlighten layer. It's time to wrap it in IR LEDs. You may want to consider purchasing some U-bars to give the table a cleaner feel and look. As for us, since we were still testing, we used Duct Tape again to hold the IR in place. All you have to do is simply cut the IR reel to size and wrap it around the edges of the Acrylic. Next, connect to a power supply and verify that they work. You may not be able to see the IR wavelength with your eye, but if you use your phone, you will see an orange-yellowish glow.

I. Layering:Now we will talk layering. This step is very important because depending on how you do this, your setup will yield differences in input detection, ghosting, and transparency. We must find a way to balance these three perfectly.

• ghosting refers to the "naturalness" of your interaction with the table. We say there is too much ghosting when the Acrylic layers are too thick or ordered in a way that your touch doesn't seem to feel right. If your touch layer is, for example, an inch from the projection layer, the effect of refraction will be heavier and you will feel as if your touches are off. This is a problem in that you won't get the full touchscreen experience.
• As far as input detection goes, what we are concerned with now is the positioning to yield maximum camera detection. There is a threshold where the layering is too off, prohibiting you from properly causing any photon collisions. When you press on the Acrylic, you force photons to collide and cause "explosions" in a way. Those explosions are termed and read as "blobs" by the software. If, for example, your resistant layer is too far from the camera or the IR layer, there will be no collisions read by the software.
• Transparency refers to the user's ability to see through the LCD. This can be used either creatively or it can be bad. The line between right and wrong is mainly blurred on this issue.

II. Example setups : I will now share the setups we experimented with and allow you to pick the best choice:

* Protective refers to abrasion resistant acrylic
*IR refers to Endlighten acrylic
*Support refers to extruded FF acrylic

1. The original plan:

-------Protective (3-5 mm)----------
=======IR( 8-10 mm)========
~~~~~~~LCD (2 mm)~~~~~~~~~
**********Diffuse( 1-2 mm)**********
_______Support (8 mm)________
.
.
.
. ~3 ft
.
.
.
_________|--camera--|___________

**Report: Heavy ghosting occurred due to the support being so thick. Also, the collisions in the IR wouldn't escape the 10 mm piece through the diffuse layer. Barely got blobs registered unless the the user increased pressure of touch (which could harm the integrity of the system) AND removed the protective layer. Also, the distance between the camera and layers was too great. Discovered that it works better if the camera distance was reduced, the thickness of the IR and support were reduced:

-------Protective (3 mm)----------
=======IR( 8 or less mm)==
~~~~~~~LCD (2 mm)~~~~~~~~~
**********Diffuse( 1-2 mm)**********
_______Support (<8 (maybe 3) mm)________
. ~ 1-2 ft
_________|--camera--|___________

This seems to be what will work best.

2. The next setup worked due to the sturdiness of the box:

=======IR( 8 mm)========
~~~~~~~LCD (2 mm)~~~~~~~~~
**********Diffuse( 1-2 mm)**********
_______Support (3-5 mm)________
.
.
.
.
.
~3ft
.
.
.
.

_________|--camera--|___________

***Report: Blobs were present but faint. some degree of pressure was needed to register input. Also, due to the sturdiness of the box, we did not have to worry about the layers warping. The initial distance tested was 3 ft. We assume that if the distance was decreased to 1.6 feet, the blobs would be read perfectly as the lens we purchased agreed to this mathematically. Only major problem would be the exclusion of the protective layer, yet the Endlighten fared to be unexpectedly durable so long as we occasionally wiped it down. I would recommend this setup.

3. Creative setup: This is my personal favorite:

=======IR( 8 mm)========
~~~~~~~LCD (2 mm)~~~~~~~~~
_______Support (3-5 mm)________
~1-2 ft
_________|--camera--|___________

**Report: Perfect blobs read. Additionally, decreasing the size from 3 ft to 1.6 ft would only make it better (allowing for us to place a protective layer). The downside is that since the diffuse layer has been removed, the transparency increases. We sacrifice transparency for perfect detection and virtually no ghosting. The transparency is not bad when the is no ambient light ( it is non intrusive). If the user wishes to use the table in an area with ambient light, this setup is not recommended as they will not see the screen performing at maximum quality. If the user is creative, they can show off the guts of the system and make it very presentable. You can attempt this:

-------Protective (3 mm)----------
=======IR( 8 or less mm)==
~~~~~~~LCD (2 mm)~~~~~~~~~
_______Support (3 mm)________
. ~ 1-2 ft .
_________|--camera--|___________

The above is the example of the setup that we are moving toward at the moment for experimentation.

4. Setup that needs more exploring:

=======IR( 8-10 or less mm)==
~~~~~~~LCD (2 mm)~~~~~~~~~
. ~ 1-2 ft .
_________|--camera--|___________

**Report: Not recommended unless a support layer is added (LCDs are very fragile), but it is an interesting setup. received perfect blobs, no ghosting, yet great deal of transparency. Perhaps adding a layer of support and a thin diffuse layer will work.

III. Comments

A. The diffuser sheet we used was the thicker one that is packaged with the LCD as my teammate accidentally threw away the smaller ones. I am fairly certain that the thinner sheets would have worked best to reduce transparency in the case of the setups we built without diffuse layers.

B. I highly encourage the builder to read through other projects accomplished on the NUI group forums and formulate a design plan before starting the project.

C. I did not feel that I needed to thoroughly explain the re-assembly for the LCD panel onto the box. All that you have to do is place it down on top of your support layer and reconnect the appropriate FFC to the sockets on the T-con Board. It is important that you connect them to the sockets they correspond to. Also, make sure the sockets are clean and that the FFC are in correctly. If you place them backwards or incorrectly, you may see a column of dead pixels. This is nothing to be too alarmed about as you only need to fix the connection.

Step 10: Did I break it? Testing the TV (Hardware/Information)

Once you are content with a specific layering, turn on the TV and watch something (or use your computer). Make sure that you have a mouse and keyboard as you are not yet able to receive responses for your inputs. Also, make sure that your white LEDs are on (doesn't matter if your IR LEDs are on for this step). If you can see as we did in the picture above, then all is well and you should congratulate yourself on being one of the few people to re-purpose an LCD and have it work on the first try!

I. Common troubleshoots:

1. Twice-inverted letters and picture? Your LCD is backwards, simply flip the panel
2. No picture? Check that the HDMI or video input of your choice is properly configured. Also, if you kept the LED indicator, check it to tell whether the TV should be on or not. you will notice that the range of the TV remote just increased by 7 times. You are welcome.
3. Screen is black? Turn on white LEDs. If LEDs are on, make sure they are white and not IR. If still no picture, your Flat Flex Cables are connected in the wrong order.
4. No sound? your speakers are connected on the wrong socket.

**For those wondering, the picture above is the system running Ubuntu 13.04 "Raring." This was the first OS we decided to run.

Step 11: Understanding the software (Information)

We will be running a variety of warez for this project to work. Luckily for us, the NUI community has already developed an open source version of the tracking software in order for the controller to properly determine whether an instance is a touch event or not. This is mostly lucky in that there are many tracking suites out there that go for a good $3,000 or so. The program we will be using, CCV, runs equally as great and is much easier to troubleshoot. This, however, does not mean that it is an easy process. I will start by emphasizing that you will come across many problems while attempting to run CCV. Your system may require you to do different things in order to get it working. Also, if you are running Windows 8.1 (although it would be an appropriate OS for multi-touch) , the documentation is very scarce. The reason you will rarely find any information regarding software setup is that many of us who get it to work have done so by some stroke of luck. Depending on your machine, you may also need to install the different packages in a certain specific order.

I. How the tracking software works:

The algorithm for tracking and maintaining touch events is a bit complex, yet very fascinating. Keep in mind that although Jeff Han pioneered the use of IR technology in his Frustrated Total Internal Reflection (FTIR) multi-touch displays just recently, the software techniques have been used for a long time.

The process involves a set of equations:

• The state equation X_k = X_(k-1) + VU_k +W_k
  • Where X_k describes the propagation of the state variable for position X, X_(k-1) is the true position at time = k-1 of the event, W_k is a noise term, and VU_k is a velocity term of constant velocity V.
• The observation equation Y_k = X_k + n_k
  • where n_k is the zero-mean Gaussian white noise.

Step 12: Choosing an OS (Information)

Your input is going to heavily depend on your operating system of choice. Each OS delivers a very different experience; thus, it is hard to say which is the best. For this section, I can only give a breakdown of a couple of OS' that I know and let you make your choice from there. This step IS important in that your method of capture (software-wise) will depend on the OS you choose.

I. OS Breakdown:

1. Microsoft's Windows
   1. Windows 8/ Windows 8.1/RT: Windows 8, RT, and 8.1 were optimized for the touchscreen apparatus. The metro offers a rich experience that feels like it was created specifically for a multi-touch device. Also, the corner sliding and huge amount of touchscreen apps already available through the windows store offer this OS a great advantage. We personally chose this OS because it was the easiest to set up. The NUI group; however, does not have too much documentation on this OS as it is not the preferred Windows OS. Also, a couple of changes in the coding made two library files be missing in this update--this only means that you'll find yourself downloading additional files an databases in order to compensate. Also, you must choose whether to overlay or keep the built-in touch system. The NUI community will push for the use of Touch Injector as a service to give the inputs functions rather than the established MT Vista. Also, since the three versions were written differently, the setup may be different. There are blogs whose users have modded the appropriate software in order to be optimized for your version. You may encounter problems such as no input being shown, the touch-tracker not showing up on start-up, and permission revocation. You must be the account's administrator and run the warez in this mode in order for things to work properly. Lastly, since the warez is open-source for the most part, you may have conflicts with your anti-virus application. Simply treat these as false positives and move on.
   2. Windows 7: It is smooth, fast, and perfect for multi-touch. Similar to the above version, you should try to build your setup in a way that the corners are free for touching as the start menu and the important apps for this are located on the task bar's corners by default. You will need to install a number of services in order to get your input recognized by the controller, but this has the most documentation to any other OS in the NUI forums. You should not have many problems dealing with this OS when it comes to multi-touch. Also, if you are a student, you may have access to the Microsoft Surface Developer Kit which will increase the functionality of you table. You can also experiment with building your own frameworks and graphical interfaces if you so desire--which is always a great thing. If you are an avid windows fanatic, I would recommend this OS as it is a blessing.
   3. Windows Vista/ XP: I grouped Vista with XP since after the service packs released for these OS', XP and Vista simply become Windows pre-7.1. I do believe their service has been discontinued (don't quote me on that) so I would not recommend these if you are worried about a longer-term experience with your table--I've not yet attempted to upgrade or downgrade an OS with the multi-touch Warez already installed. Other than that, you'll be finding a good amount of support in the NUI forums due to the newer software simply being a modded version of software originally for XP and Vista. You shouldn't have too many problems with these, but i recommend not to go here unless you are planning to run a virtual machine with a more updated OS--most apps have been optimized for the more recent OS' and I'd recommend starting at 7 if you want that old-fashioned Windows appeal.
   ** Side Note: Consider installing a program named Rain-meter. Rain-meter takes the widgets introduced in vista to a new level by allowing you to set up your computer exactly how you'd like it. You can create futuristic displays or even have sound output given for certain actions. This app is compatible with most Windows OS versions as far as I am aware. I use it last on my personal Win 8 laptop setup. Ubuntu offers an applet for this named Conky. If you use Conky, I advise downloading Conky Manager.
2. Linux OS:
   1. Canonical's Ubuntu (any release from 12.04 LTS to 13.10): When it comes to Linux OS', people seem to think you must be a hacker, or an overly obsessed computer maniac in order to understand what is going on--the movies seem to portray Linux as a completely text-based OS, and who wants that? Well, let me be the first to tell you that they are wrong! Ubuntu is a FREE (although I'd recommend leaving a donation to the company when downloading) OS distributed by Canonical that is beyond wonderful. It is classified under the "Debian-like" distributions as this is where it is sourced from. Ubuntu is virtually updated daily and provides the highest sense of security for the average user. It handles similarly to Windows, yet it has it's own advantages that a Windows OS could not provide you. A big pro is that you won't find your system being infected by viruses. Although you can still contract Trojans and such, your system will be unharmed as the viruses are written for Windows users--they won't open and execute on Ubuntu or any Linux distro (unless you use WINE, a windows emulator for Linux). If you dual boot with Windows, you can even use Ubuntu to rid your Windows partition of any infections it may have. Ubuntu 13.10 (dubbed "Saucy Salamander") was written with touchscreen devices in mind much like windows 8; however, the format of the graphical interface is non-intrusive, and the options are all equally optimized for the non-touch aficionado. Paired with a variety of apps that you can find in their software center, you can make this desktop exactly what you want it to be, giving you the ultimate freedom and maneuverability. So what's the problem with it then? It is very hard to find any documentation on setting up an Ubuntu multi-touch setup on the NUI forums. This is due to many of the programs that are needed for touch implementation being written with Windows users in mind. You'll have to find a replacement capture software--I've heard Cheese or VLC Media Player work well with Ubuntu in place of the CLeye for the PS3 eye, but you WILL have to experiment and go through a lot of hard work in order to get this setup to work. The few that I've seen are gorgeous, so it IS worth the trouble. I am personally planning on getting my table back soon in order to experiment with changing the OS back to Ubuntu. Also, while on Ubuntu, you'll find yourself dealing with the Terminal (Linux equivalent of the Command Prompt console or Powershell) more than anything else. Although everything gives you the option of graphical input, you may find the terminal to be more satisfying. If you want to learn about computers, processes and all that goes into providing a rich, safe, and efficient user experience, I'd say give any Linux distro a chance.
      ****You should probably keep in mind that although you can find a Linux replacement for any Windows program, you're not going to be Gaming too much on a Linux Machine. Most developers and gamers prefer the Windows setup for gaming.
   2. Mint, Tails, Kubuntu, Xubuntu, Debian, or any of the other hundreds of Linux distros: I know I grouped a lot of OS' into one category--which should be a cardinal sin as each one handles and behaves differently. The issue is that I've only merely dabbled in these other OS' and haven't given myself much time to get fully acquainted with any other distro. Kubuntu and Xubuntu offer a more simplistic experience to Ubuntu and Tails is optimized for anonymity. I've not really heard of any members in the NUI forums dealing with these specifically, but I know that it can be done. I'm not able to fully give an opinion on these as I have not been so involved with them, but I have heard that Mint OS is a pretty good choice for those wanting to experience the feel of Linux.
3. Apple OS'
   1. I've personally never dealt with an Apple OS. I hear that the are Unix-like and have been developed from Unix, but I've never owned an Apple machine. However, there is some documentation on the NUI Forums and on the internet regarding Apple's many cat-themed OS'. I hear that a setup with these OS' runs similar to a desktop rather than an iPad, unless you can configure it in that manner. I am unfortunately uninformed as to the warez necessary or anything of the like regarding apple. It's not that I don't like them, but I've never had one so I cannot formulate a fair opinion on any of them. I do, however recognize the security of the software implemented in their devices though. Unlike Linux, you may still encounter some minor infections due to the code not being updated as frequently, but I'd say that for the regular user (not trying to access anything dangerous), this is a great choice.

So there you have it. Now all you have to do is choose an OS and get what you need to work things out. If you are not too much into computers, I'd recommend that you stick with the Windows setup (maybe Win 7). If you want a challenge--and want everything to be worthwhile (and don't want to spend money on an OS)--go with a Linux OS. If you are an Apple user, go with Apple!

**If you really want something interesting though, try investing some time into setting up an Android setup. Android is Unix-like so there have been some developers that have succeeded. Also, the developer kit is free and there is a rich community of developers for Android. You may have a hard time at first, but it should pay off in the end. Just an interesting thought for you.

Step 13: CCV, Touch Injector and the Cleye + CCV tests (Warez)

Now that we chose an OS, let's talk about what we need to get things working for us. Keep in mind that for our setup, we decided to go for a Windows 8 setup as it was freely available to us as a donation from our software guy and since it was also the easiest for him to develop for under the time we had set out for the project. This is the warez you will need to make touch work:

I. CCV:

CCV stands for Community Core Vision, the open-source and NUI group-developed tracking software at the basis for any multi-touch table you will find to be built in the NUI group forums. It comes with its own handy interface that luckily does not change too much in between version revision. I believe the CCV distro we are currently in is 1.5--this is the one we used since it has documentation and support for multi-camera setups. There is a mod in the NUI forums for 1.4 that allows it to accept multi-camera setups. The more cameras, the faster your table and the smaller you can build it. I've explained in a very layman's terms how this program works in the Understanding Software section. There are various replacements to this program, although I do not recommend going that route as the NUI forums specifically highlight CCV over all other programs.

The interface can be quite confusing to deal with. Seth Sandler, a contributor to the NUI group has posted a blog with a great description of the interface and how to navigate it here. Try to read through his site and discover more on the warez needed. I believe his is optimized for a Windows 7-based MT Mini--but the practicality is still essentially the same.


II. CLeye:

The Code Laboratories SDK for the PS3 eye. It comes with a variety of drivers and tools to sync and configure the PS3 eye to your setup via the computer. If you are working with CCV 1.5, I recommend downloading this first and making sure that your input can be read through this program. The graphical interface should display the capture of your blobs as you touch the table. Once you've got this done, you can proceed to CCV capture. There is a problem with registration regarding the CLeye though. The SDK requires that you buy a license for ~$35 in order to get the software to work. Luckily, there are a couple of ways to circumvent this; however, you will be making things harder for yourself as the process requires the handling and manipulation of DLL files. This could potentially mess up your software setup and force you to have to re-install everything all over again many times. You can find the drivers for the CLeye to be detailed on the main CCV 1.5 forum post given in the CCV section or here, and here is the CLeye site.

III. Touch Injector:

Since we're on Windows 8, we have to find a replacement for MT Vista that will work as a service for your inputs. For this, we use Touch Injector. This was also developed in the NUI group and handles very well. It also has quite a bit of documentation and there are tons of forum posts on debugging and troubleshooting..

Step 14: Calibration (Software)

Assuming that your blobs are perfect, and that you are satisfied with the framerate and all other factors, the final thing to do is to calibrate CCV settings. Once again, Sandler has provided an amazingly detailed guide on the steps for proper calibration. I advise that you go read it here. Once you are done with this step--if successful--you have essentially finished the construction of your setup. Your efforts and money should have (hopefully) paid off, and you will be the proud owner of an LCD-based DSI touch screen table that is fully functional and pleasing.

Step 15: Test run (Software)

The best way test your setup? Flash demos of course! for the last time and download some demo applications from NUI folders. Or simply head to the windows store and download a game. If you encounter some problems, you are most likely to find a solution to it via the forums so keep those in mind. Either way, have fun with your new table!

Step 16: Done!!! (Happiness)

So you've finally completed your table. Feel free to leave any feedback on how this instructable could be made better. Also, feel free to post videos and pics of your setup in the comments below. I'll be sure to add more pics and a couple of videos once I get my setup modded.

If you realized anything from this instructable, let it be this: Multi-touch isn't just a toy, nor is it a technology that we should take too much for granted. Projects like these are gateways to the future of NUI technologies. Multi-touch facilitates the branching toward the naturalization of the human to computer interface. Children are able to become more engaged and willing to learn about technology in this manner--as we are a species that yearns for feedback from our senses--and the elderly and disabled peoples can find more comfort and relief in navigating these devices. More importantly, Multi-touch opens the door to greater technologies such as Pranav Mistry's SixthSense gestural interface (an instructable that may make it's way sooner than you may think) and a host of other interface revisions--some of which I am not able to fully reveal as I am planning on developing. The fact that you've completed this project is truly something to boast about.
**Update: I wanted to take a moment to thank all my viewers and subscribers for voting and allowing me to win my first contest! Can't wait to start on my next large-scale project--after I do some more finishing touches on this one of course.
tags - #surface computing #touch screen #touch table #multitouch table #nui #diy