谷歌核心技术教学提纲.ppt

《谷歌核心技术教学提纲.ppt》由会员分享，可在线阅读，更多相关《谷歌核心技术教学提纲.ppt（32页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、谷歌核心技术The InternetFrom Hardware to CommunityThe Internet:From Hardware to CommunityMySpaceFacebook开心网校内网What Do Todays Users Want?AccessibilityAccess from anywhere and from multiple devicesShareabilityMake sharing as easy as creating and savingFreedomUsers dont want their data held hostageSimplicity

2、Easy-to-learn,easy-to-useSecurityTrust that data will not be lost or seen by unwanted partiesThe InnovationA Computing CloudCloud ComputingAttributes of Cloud ComputingData stored on the cloudSoftware&services on the cloud-Access via web browserBased on standards and protocols-Linux,AJAX,LAMP,etc.Ac

3、cessible from any deviceHardware CentricSoftware CentricService CentricPersonal PCClient ServerCloud ComputingBreakthroughs for Cloud ComputingBreakthroughs for Cloud ComputingUser-Centric1Task-Centric2Powerful3Intelligent4Affordable5Programmable6User CentricData stored in the“Cloud”Data follows you

4、&your devicesData accessible anywhereData can be shared with othersmusicpreferencesmapsnewscontactsmessagesmailing listsphotoe-mailscalendarphone numbersinvestmentsExample:GMailJust a web browser and your account with password!Once you login,the device is“yours”.Data stored on remote servers in the“

5、cloud”(with large capacity)Beijing,on travelSan Francisco,MondayHome,WednesdayUse Google Docs to Solve a TaskAccess your docs from anywhereChat with others in real timeChanges instantly appear to other collaboratorsTask=“Teachers creating a departmental curriculum”Communication Task Email,Chat,Conta

6、cts,Chat HistoryTask:Collaborate on Spreadsheet Communicate Chat with others editing the spreadsheetTask:Collaborate on Spreadsheet CollaborateInvite others to collaborate on the spreadsheetTask:Collaborate on Spreadsheet Publish Invite others to view the spreadsheetYou can also easily organize all

7、your common tasks Cloud Computing is Powerful:It can do what no PC can doIs Google Search faster than search in Windows/Outlook/Word?And Google Search must be much harder.How much storage does it take to store all of the web pages?100B pages*10K per page=1000T disk!Cloud computing has at its disposa

8、lEssentially infinite amount of diskEssentially infinite amount of computation(Assuming they can be parallelized)Example:Google SearchWeb Page Search Universal SearchW1st Generation:era of single search not diverse2nd Generation:era of vertical search too complex3rd Generation:an era of Universal Se

9、archABCDEFrom vertical search to universal searchABCDEIntegration of user experienceUniversal Search ExampleUniversal Search ExampleCloud Computing Infrastructure24GFS ArchitectureGoogle48%MSN19%Yahoo33%Files broken into chunks(typically 64 MB)Master manages metadataData transfers happen directly be

10、tween clients/chunkserversClientClientClientReplicasMastersGFS MasterGFS MasterC0C1C2C5Chunkserver 1C0C2C5Chunkserver NC1C3C5Chunkserver 2ClientClientClientClientClientClientTypical Cluster25Scheduling mastersGFSchunkserverSchedulerslaveLinuxMachine 1User app2Userapp1GFS masterLock serviceGFSchunkse

11、rverSchedulerslaveLinuxMachine NUserapp3User app2Userapp1GFSchunkserverSchedulerslaveLinuxMachine 2Userapp3MapReduce26More specifically27Programmer specifies two primary methods:map(k,v)*reduce(k,*)*All v with same k are reduced together,in order.Usually also specify:partition(k,total partitions)-pa

12、rtition for koften a simple hash of the keyallows reduce operations for different k to be parallelized28BigTableDistributed multi-level mapWith an interesting data modelFault-tolerant,persistentScalableThousands of serversTerabytes of in-memory dataPetabyte of disk-based dataMillions of reads/writes

13、 per second,efficient scansSelf-managingServers can be added/removed dynamicallyServers adjust to load imbalance29BigTable:Basic Data ModelDistributed multi-dimensional sparse map(row,column,timestamp)cell contentsGood match for most of our applications“”ROWSCOLUMNSTIMESTAMPS“contents”BigTable:Syste

14、m ArchitectureCluster Scheduling Masterhandles failover,monitoringGFSholds tablet data,logsLock serviceholds metadata,handles master-electionBigtable tablet serverserves dataBigtable tablet serverserves dataBigtable tablet serverserves dataBigtable masterperforms metadata ops,load balancingBigtable cellBigtable clientBigtable clientlibraryOpen()ThanksQ&A此此课课件下件下载载可自行可自行编辑编辑修改，修改，仅仅供参考！供参考！感感谢谢您的支持，我您的支持，我们们努力做得更好！努力做得更好！谢谢谢谢