本科毕业设计-kinect-windowshardware和sdk概述--外文文献翻译.doc

外文翻译Kinect for WindowsHardware and SDK OverviewIn this chapter we will define the key notions and tips for the following topics: Critical hardware components of the Kinect for Windows device and theirfunctionalities, properties, and limits Software architecture defining the Kinect SDK 1.6Motion computing and Kinect.Before getting Kinect in motion, let's try to understand what motion computing(or motion control computing) is and how Kinect built its success in this area.Motion control computing is the discipline that processes, digitalizes, anddetects the position and/or velocity of people and objects in order to interact withsoftware systems.Motion control computing has been establishing itself as one of the most relevanttechniques for designing and implementing a Natural User Interface (NUI).NUIs are human-machine interfaces that enable the user to interact in a natural waywith software systems. The goals of NUIs are to be natural and intuitive. NUIs arebuilt on the following two main principles: The NUI has to be imperceptible, thanks to its intuitive characteristics:(a sensor able to capture our gestures, a microphone able to capture ourvoice, and a touch screen able to capture our hands' movements). All theseinterfaces are imperceptible to us because their use is intuitive. The interface is not distracting us from the core functionalities of our software system.Kinect for WindowsHardware and SDK Overview The NUI is based on nature or natural elements. (the slide gesture, the touch,the body movements, the voice commandsall these actions are natural and not diverting from our normal behavior).NUIs are becoming crucial for increasing and enhancing the user accessibility forsoftware solution. Programming a NUI is very important nowadays and it will continue to evolve in the future.Kinect embraces the NUIs principle and provides a powerful multimodal interface to the user. We can interact with complex software applications and/or video games simply by using our voice and our natural gestures. Kinect can detect our body position, velocity of our movements, and our voice commands. It can detect objects' position too.Microsoft started to develop Kinect as a secret project in 2006 within the Xbox division as a competitive Wii killer. In 2008, Microsoft started Project Natal, named after the Microsoft General Manager of Incubation Alex Kipman's hometown in Brazil. Theproject's goal was to develop a device including depth recognition, motion tracking,facial recognition, and speech recognition based on the video recognition technologydeveloped by PrimeSense. Kinect for Xbox was launched in November 2010 and its launch was indeed asuccess: it was and it is still a break-through in the gaming world and it holds theGuinness World Record for being the "fastest selling consumer electronics device"ahead of the iPhone and the iPad.In December 2010, PrimeSense () released a set of open sourcedrivers and APIs for Kinect that enabled software developers to develop Windows applications using the Kinect sensor. Finally, on June 17 2011 Microsoft launched the Kinect SDK beta, which is a set of libraries and APIs that enable us to design and develop software applications on Microsoft platforms using the Kinect sensor as a multimodal interface. With the launch of the Kinect for Windows device and the Kinect SDK, motion control computing is now a discipline that we can shape in our garages, writing simple and powerful software applications ourselves. This book is written for all of us who want to develop market-ready software applications using Kinect for Windows that can track audio and video and control motion based on NUI. In an area where Kinect established itself in such a short span of time, there is the need to consolidate all the technical resources and develop them in an appropriate way: this is our zero-to-hero Kinect in motion journey. This is what this book is about.This book assumes that you have a basic knowledge of C# and that we all have a great passion to learn about programming for Kinect devices. This book can be enjoyed by anybody interested in knowing more about the device and learning howto track audio and video using the Kinect for Windows Software Development Kit (SDK) 1.6. We deeply believe this book will help you to master how to process videodepth and audio stream and build market-ready applications that control motion.This book has deliberately been kept simple and concise, which will aid you toquickly grasp the core and critical concepts.Before jumping on the core of audio and visual tracking with Kinect for Windows,let's take the space of this introduction chapter to understand what the hardware and software architectures Kinect for Windows and its SDK 1.6 use.Hardware overviewThe Kinect device is a horizontal bar composed of multiple sensors connected to a base with a motorized pivot.The following image provides a schematic representation of all the main Kinect hardware components. Looking at the Kinect sensor from the front, from the outside it is possible to identify the Infrared (IR) Projector (1), the RGB camera (3), and thedepth camera (2). An array of four microphones (6), the three-axis accelerometer (5),and the tilt motor (4) are arranged inside the plastic case.The device is connected to a PC through a USB 2.0 cable. It needs an external power supply in order to work because USB ports don't provide enough power.Now let's jump in to the main features of its components.The IR projectorThe IR projector is the device that Kinect uses for projecting the IR rays that are used for computing the depth data. The IR projector, which from the outside looks like a common camera, is a laser emitter that constantly projects a pattern of structured IR dots at a wavelength around of 830 nm (patent US20100118123, Prime Sense Ltd.).This light beam is invisible to human eyes (that typically respond to wavelengths from about 390 nm to 750 nm) except for a red bright dot in the center of emitter.The pattern is composed by 3 x 3 subpatterns of 211 x 165 dots (for a total of 633 x 495 dots). In each subpattern, one spot is much brighter than all the others.As the dotted light (spot) hits an object, the pattern becomes distorted, and thisdistortion is analyzed by the depth camera in order to estimate the distancebetween the sensor and the object itself.Depth cameraThe depth camera is a (traditional) monochrome CMOS (complementarymetal-oxide-semiconductor) camera that is fitted with an IR-pass filter(which is blocking the visible light). The depth camera is the device thatKinect uses for capturing the depth dataThe depth camera is the sensor returning the 3D coordinates (x, y, z) of the scene as a stream. The sensor captures the structured light emitted by the IR projector and the light reflected from the objects inside the scene. All this data is converted in to a stream of frames. Every single frame is processed by the PrimeSense chip that produces an output stream of frames. The output resolution is upto 640 x 480 pixels.Each pixel, based on 11 bits, can represent 2048 levels of depth.The RGB cameraThe RGB camera is similar to a common color webcam, but unlike a common webcam, the RGB camera hasn't got an IR-cut filter. Therefore in the RGB camera, the IR is reaching the CMOS. The camera allows a resolution upto 1280 x 960 pixels with 12 images per second speed. We can reach a frame rate of 30 images per second at a resolution of 640 x 480 with 8 bits per channel producing a Bayer filter output with a RGGBD pattern. This camera is also able to perform color flicker avoidance, color saturation operations, and automatic white balancing. This data is utilized to obtain the details of people and objects inside the scene.Microsoft now announced it has started delivering Kinect for Windows v2 Developer Preview kits to developers whosigned up for the $399 program. The Developer Preview includes a pre-release Kinect for Windows v2 sensor and the next release of the Kinect for Windows software development kit (SDK).Microsoft says it received “thousands of applications” for the program and selected an unknown number (again, “thousands”) of participants based on their “expertise, passion, and the raw creativity of their ideas.” The company is promising ongoing updates as well as access to private forums where developers can discuss what works and what doesnt.The best part, however, is that participants will also receive a Kinect for Windows v2 sensor when they become available “next summer.” Microsoft wouldnt share a more specific timeframe than that.The Kinect for Windows v2 sensor will feature the core capabilities of the new Kinect for Xbox One sensor, including an ultra wide-angle 1080p HD depth camera. The device captures RGB color video at around 30 frames per second and can even monitor the users heart rate.Here is how the company breaks down the improvements:Real Vision: Kinect Real Vision technology dramatically expands its field of view for greater line of sight. An all-new active IR camera enables it to see in the dark. And by using advanced three-dimensional geometry, it can even tell if youre standing off balance.Real Motion: Kinect Real Motion technology tracks even the slightest gestures. So a simple squeeze of your hand results in precise control over an application, whether youre standing up or sitting down.Real Voice: Kinect Real Voice technology focuses on the sounds that matter. Thanks to an all-new multi-microphone array, the advanced noise isolation capability lets the sensor know who to listen to, even in a crowded space.It beats the original Kinect on almost every level. Microsoft believes its Natural User Interface (NUI) benefits will impact multiple industries, from retail and manufacturing to healthcare, education, communications, and so on. We dont doubt it.See also  Microsoft updates Kinect for Windows SDK with background removal, color capture, other new APIs and samples and Microsoft releases three videos showing off how Kinect for Windows can work for retailersTo install the SDK:Make sure the Kinect sensor is not plugged into any of the USB ports on the computer.If you have a previous version of the Kinect for Windows SDK currently installed, close any open samples, the Sample Browser, etc. and skip to step 5. Kinect for Windows v1.8 will upgrade the previous version.Remove any other drivers for the Kinect sensor.If you have Microsoft Server Speech Platform 10.2 installed, uninstall the Microsoft Server Speech Platform Runtime and SDK components including both the x86 and x64 bit versions, plus the Microsoft Server Speech Recognition Language - Kinect Language Pack.Close Visual Studio. You must close Visual Studio before installing the SDK and then restart it after installation to pick up environment variables that the SDK requires.From the download location, double-click on KinectSDK-v1.8-Setup.exe. This single installer works for both 32-bit and 64-bit Windows.Once the SDK has completed installing successfully, ensure the Kinect sensor is plugged into an external power source and then plug the Kinect sensor into the PC's USB port. The drivers will load automatically.The Kinect sensor should now be working correctly.Download the Kinect for Windows Developer Toolkit, which contains source code samples, tools, and other valuable development resources that simplify developing Kinect for Windows applications.Kinect Windows-Hardware和SDK概述在这里,我们将定义的关键概念和技巧分为以下部分:Kinect的关键硬件组件为Windows设备及其功能属性和限制软件定义架构的Kinect SDK 1.6运动计算和Kinect。Kinect在运动之前,让我们试着了解运动计算(或运动控制计算)和Kinect如何在这一领域获得的成功。运动控制计算实现的过程，是人和物体的位置或速度以软件系统交互。运动控制计算已经建立本身作为最先进设计和实现一个自然的用户界面(NUI)。NUI是人机接口,允许用户在自然软件系统下进行交互。目标NUI很自然而且直观。NUI界面有以下两个主要原则:由于其直观的特点:(一个传感器可以捕捉到我们的手势,一个麦克风能够捕捉我们的声音,触摸屏能抓住我们手的动作)。所有这些接口是无法察觉的,因为其使用是直观的。接口不是分散的核心功能的软件系统。Kinect和SDK概述: 努伊是基于自然的元素。(进行滑动的动作时,或触摸身体动作、声音这些行动是自然的正常转移行为)。NUI正变得至关重要的提高和增强用户的可访问性解决方案。现在编程一个NUI, 在未来的发展中它将是非常重要的。Kinect拥有NUI原理和一个强大的多通道界面。只需使用我们的声音和自然的手势，我们就可以与复杂的软件应用程序或视频游戏进行交互。Kinect可以检测我们的身体位置、速度的运动,和我们的语音命令。它也可以检测到对象的位置。微软开始发展Kinect是在2006年，当时是一个神秘的项目Xbox Kinect。2008年,微软开始以Project Natal命名，译为初生的意思，当时项目的目标是开发一个设备包括深度识别、运动跟踪、面部识别,基于视频识别技术和语音识别技术的Xbox Kinect 。2010年11月建立项目,它拥有吉尼斯世界纪录“最快销售消费电子装置”领跑在iPhone和iPad的前面。2010年12月,微软公司发布了一套开放资源的Kinect,使软件开发人员使用Kinect传感器开发Windows应用程序。最后,2011年6月17日微软推出了Kinect SDK beta版,使人们能够设计和开发软件应用程序在微软平台上使用Kinect传感器作为多通道界面。Kinect为Windows的启动设备 , 使人们可以编写简单而强大的软件应用程序。这本书是写给所有想开发市场化的人使用的,Kinect SDK beta版可以跟踪基于NUI音频和视频的控制运动。Kinect在一个领域确立了的完整体系,但是还需要巩固所有的技术资源和发展在一个适当的方法:这是我们接下来的zero-to-hero Kinect运动旅行。1.6开发工具包(SDK)。我们深深相信这本书将帮助人们掌握如何处理音频流,建立市场化运动控制的应用程序。这本书却一直刻意保持简单和简洁,这将帮助你很快的抓住核心和关键概念。了解硬件和软件架构Kinect为Windows SDK 1.6使用做好准备。概述:Kinect设备是由多个传感器组成的单杠机动枢轴。看着Kinect传感器的前面,从外面可以识别(1)红外(IR)投影仪(2)RGB相机和深度摄像头(3)一个数组的四个麦克风(4)三轴加速度计(5)和电机。倾斜排列在塑料盒内。设备连接到电脑通过USB 2.0插孔。为了维持工作它需要一个外部电源,因为USB接口不提供足够的电力。 现在,让我们看一下组件的主要特点。深度相机:深度相机(传统)单色CMOS(互补)相机,装有一个IR-pass过滤器(这是阻止可见光)。深度摄像头的作用是Kinect用于获取深度数据，即深度相机传感器返回的3D坐标(x,y,z)场景。传感器捕获结构光发射的红外投影仪和场景内物体反射的光线。所有这些数据转换成一连串的帧。处理每一帧数据,产生一个输出流的帧。输出分辨率高达640 x 480像素。每个像素基于11位,能代表2048年的水平的深度。RGB相机:RGB相机是类似于常见的彩色摄像头,但与一个共同的摄像头,RGB相机没有IR-cut过滤器。因此RGB摄像机允许与12图像，分辨率高达1280 x 1280像素每秒。可以达到每秒30帧率图像的分辨率640 x 480每通道8位与拜耳滤波器输出模式。这款相机还能够执行避免颜色闪烁,色彩饱和度操作和自动白平衡。这个数据是用来获得人和物体所在场景的细节。因为USB接口不提供足够的工作能力。微软现在宣布,它已经开始交付Kinect为Windows v2开发者预览工具开发者whosigned的计划，价格为399美元。开发者预览版包括一个预发布Kinect，它为Windows v2传感器和Kinect的下一版本Windows软件开发工具包(SDK)。微软说它收到了“成千上万的应用程序”的信函,选择了未知号码(再一次,“数千”)，是参与者根据他们的“专业、热情和原始创造力的想法。”该公司承诺持续更新以及获得私人论坛,开发者可以讨论其可行性。然而,最好的部分是,参与者也将得到一个Kinect为Windows v2传感器,最早明年夏天即可使用。但微软没有分享更具体的时间表。Kinect为Windows v2传感器，它为新版Xbox的传感器,包括一个超广角1080 p高清深度相机。设备捕获RGB颜色视频在每秒30帧,甚至可以监视用户的心率。以下是公司具体的改进措施。真正的愿景:Kinect真正视觉技术极大地扩大视野更大的视线。一个全新的主动红外相机使其在黑暗中看到。并利用先进的三维几何,它甚至可以告诉你站立是否会失去平衡。真正的运动:Kinect真实运动技术跟踪即使是最轻微的动作。这样一个简单的挤压，你的手会导致应用程序精确控制,无论你是站着或坐着。真实的声音:Kinect真实声音技术关注的声音。由于一个全新multi-microphone数组,高级噪音隔离功能让传感器知道听谁,即使是在一个拥挤的空间。它比原来的Kinect在几乎每一个级别。微软认为其自然用户界面(NUI)将影响多个行业,从零售和制造业到医疗、教育、通讯等等。我们对此深信不疑。微软Kinect为Windows SDK更新背景,颜色捕捉,其他新的api和样本和微软发布三个视频展示Kinect为Windows如何为零售商工作。安装SDK:确保Kinect传感器是没有插入任何计算机上的USB端口。如果你有以前版本的Kinect目前Windows SDK安装,关闭任何打开的样本,样本浏览器等,跳过步骤5。 Kinect为Windows v1.8将升级之前的版本。删除任何其他司机的Kinect传感器。如果你安装了微软服务器言论平台10.2,卸载微软服务器言论平台运行时和SDK组件包括x86和x64版本,加上微软服务器语音识别语言Kinect语言包。关闭Visual Studio。 之前,你必须关闭Visual Studio SDK安装,然后安装后重启接SDK所需要的环境变量。从下载位置,双击KinectSDK-v1.8-Setup.exe。 这个安装程序适用于32位和64位Windows。一旦SDK安装成功完成,确保Kinect传感器插入到一个外部电源,然后将Kinect传感器插入到PC机的USB端口。 司机将自动加载。Kinect传感器现在应该工作正常。下载Kinect为Windows开发人员工具包,其中包含源代码样本、工具和其他有价值的开发资源,简化开发Kinect为Windows应用程序。