大数据分析全英文 (16).pdf

《大数据分析全英文 (16).pdf》由会员分享，可在线阅读，更多相关《大数据分析全英文 (16).pdf（13页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、2What is Data reduction?Data reduction(subtraction)technology is used to help obtain a condensed data setfrom the original huge data set,and make this condensed data set maintain the integrity of the original data set,so that data analysis on the condensed data set is obviously efficient higher,and

2、the results of analysis are basically the same as those obtained by using the original data set.3Data reduction standardThe time spent on data reduction should not exceed or offset the time saved by analysis on the reduced dataThe data obtained by the reduction is much smaller than the original data

3、,but can produce the same or almost the same analysis results4Data reduction technology7Attributes subset selection8Attributes subset selection9Attributes subset selectionDecision tree(decision tree)inductionUse the decision tree induction method to classify and induct the initial data to obtain an

4、initial decision tree.All attributes that do not appear on the decision tree are considered irrelevant attributes.Therefore,delete these attributes from the initial attribute set to obtain an initial decision tree.A better subset of attributes.Reduction based on statistical analysisData reduction-Da

5、ta compression11Data reduction-data compressionLossless compression:Compressed data can be restored without losing any information.For example:string compression have a broad theoretical foundation and sophisticated algorithmsLossy compression:Only an approximate representation of the original data

6、can be reconstructed.For example:audio/video compressionSometimes it is possible to reconstruct a fragment without decompressing the overall data12Data reduction-data compressionPrincipal component analysis(PCA)assumes that the data to be compressed consists of N tuples or data vectors taken from k

7、dimensions.Principal component analysis and search to obtain c-dimensional orthogonal vectors that best represent the data”,where ck.In this way,the original data can be projected into a smaller space to achieve data compression.13Data reduction technology14Data reduction-Data Cube Aggregation15Data

8、 reduction-DiscretizationThree types of attribute values:Name type-e.g.value in an unordered set Ordinal-e.g.value in an ordered set Continuous value-e.g.real numberDiscretization technologyReduce the number of values of a continuous(value)attribute by dividing the range of the attribute(continuous value)domain value into several intervals.16Data reduction-concept hierarchical generationYouth Middle agedPrime of life