Big Data
Big Data is a term WPF uses to describe very large datasets and the technologies and practices of handling those datasets. Typically, Big Data datasets are so large that traditional database systems are not able to handle or analyze them.
Sources for Big Data are many and varied. They include web data, sensors, cell towers, census data and other data from the government, social media, transactional data, and a variety of other data collection systems.
We have seen a tendency to use the term Big Data as a loosely defined stand-in for a number of privacy issues that sound the same, but aren’t. For example, Big Data and Data Brokers are sometimes used together. The two ideas are distinct and different, and it is crucial for public policy and discussion that the two are not conflated as being the same thing or even a similar thing. It is possible to work with Big Data and never be a Data Broker.
Large datasets are intriguing to the World Privacy Forum, and our research on large datasets resulting from sensors and ID cards in Asia helped us understand and explore the issue in-depth. Large datasets sometimes present privacy challenges, but sometimes they do not. Much depends on how the dataflows are collected, managed, stored, and so forth. Understanding these differences and knowing when and where the challenges are is going to be important going forward in this rapidly evolving space.