AI Definitions: Big Data
/Big Data - Data that’s too big to fit on a single server, unstructured and fast-moving. In contrast, small data fits on a single server, is already in structured form (rows and columns), and changes infrequently. If you are working in Excel, you are doing small data. Two NASA researchers (Michael Cox and David Ellsworth) first wrote in a 1997 paper that when there’s too much information to fit into memory or local hard disks, “We call this the problem of big data.” Many companies end up with big data not because they need it, but because they haven’t bothered to delete the data that has become irrelevant. Thus, big data is sometimes defined as a condition where the cost of keeping data around is less than the cost of figuring out what to throw away.
Big Data serves large-scale web applications and vast sensor networks. Data science, meanwhile, looks to create AI models that capture underlying patterns in complex systems and then turn those models into working applications. Although big data and data science both offer the potential to produce value from data, the fundamental difference between them can be summarized in one statement: collecting does not mean discovering. Big data collects. Data science discovers.