Provide Data /Information In Standard Formats.
COURTESY :- vrindawan.in
Wikipedia
In the pursuit of knowledge, data (US: /ˈdætə/; UK: /ˈdeɪtə/) is a collection of discrete values that convey information, describing quantity, quality, fact, statistics, other basic units of meaning, or simply sequences of symbols that may be further interpreted. A datum is an individual state in a set of data. Data usually is organized into structures such as tables that provide additional context and meaning, and which may themselves be used as data in larger structures. Data may be used as variables in a computational process. Data may represent abstract ideas or concrete measurements. Data is commonly used in scientific research, finance, and in virtually every other form of human organizational activity. Examples of data sets include stock prices, crime rates, unemployment rates, literacy rates, and census data.
Data represents the raw facts and figures which can be used in such a manner in order to capture the useful information out of it.
Data is collected using techniques such as measurement, observation, query, or analysis, and typically represented as numbers or characters which may be further processed. Field data are data that are collected in an uncontrolled in-situ environment. Experimental data are data that are generated in the course of a controlled scientific experiment. Data is analyzed using techniques such as calculation, reasoning, discussion, presentation, visualization, or other forms of post-analysis. Prior to analaysis, raw data (or unprocessed data) is typically cleaned: Outliers are removed and obvious instrument or data entry errors are corrected.
Data has been described as “the new oil of the digital economy”. Data, as a general concept, refers to the fact that some existing information or knowledge is represented or coded in some form suitable for better usage or processing. Data is the smallest units of factual information that can be used as a basis for calculation, reasoning, or discussion. Data can range from abstract ideas to concrete measurements, including but not limited to, statistics. Thematically connected data presented in some relevant context can be viewed as information. Contextually connected pieces of information can then be described as data insights or intelligence. The stock of insights and intelligence that accumulates over time resulting from the synthesis of data into information, can then be described as knowledge.
Advances in computing technologies have led to the advent of ”Big Data”. Big Data usually refers to very large quantities of data, usually at the petabyte scale. Using traditional data analysis methods and computing, working with such large (and growing) datasets is difficult, even impossible. (Theoretically speaking, infinite data would yield infinite information, which would render extracting insights or intelligence impossible.) In response, the relatively new field of ”Data Science” uses machine learning (and other Artificial Intelligence (AI)) methods that allow for efficient applications of analytic methods to Big Data.
The Latin word data is the plural of ‘datum’, “(thing) given,” neuter past participle of dare “to give”. The first English use of the word “data” is from the 1640s. The word “data” was first used to mean “transmissible and storable computer information” in 1946. The expression “data processing” was first used in 1954.
When “data” is used more generally as a synonym for “information”, it is treated as a mass noun in singular form. This usage is common in everyday language and in technical and scientific fields such as software development and computer science. One example of this usage is the term “big data”. When used more specifically to refer to the processing and analysis of sets of data, the term retains its plural form. This usage is common in natural sciences, life sciences, social sciences, software development and computer science, and grew in popularity in the 20th and 21st centuries. Some style guides do not recognize the different meanings of the term, and simply recommend the form that best suits the target audience of the guide. For example, APA style as of the 7th edition requires “data” to be treated as a plural form.
Data, information, knowledge, and wisdom are closely related concepts, but each has its role concerning the other, and each term has its meaning. According to a common view, data is collected and analyzed; data only becomes information suitable for making decisions once it has been analyzed in some fashion. One can say that the extent to which a set of data is informative to someone depends on the extent to which it is unexpected by that person. The amount of information contained in a data stream may be characterized by its Shannon entropy.
Knowledge is the understanding based on extensive experience dealing with information on a subject. For example, the height of Mount Everest is generally considered data. The height can be measured precisely with an altimeter and entered into a database. This data may be included in a book along with other data on Mount Everest to describe the mountain in a manner useful for those who wish to decide on the best method to climb it. An understanding based on experience climbing mountains that could advise persons on the way to reach Mount Everest’s peak may be seen as “knowledge”. The practical climbing of Mount Everest’s peak based on this knowledge may be seen as “wisdom”. In other words, wisdom refers to the practical application of a person’s knowledge in those circumstances where good may result. Thus wisdom complements and completes the series “data”, “information” and “knowledge” of increasingly abstract concepts.
Data is often assumed to be the least abstract concept, information the next least, and knowledge the most abstract. In this view, data becomes information by interpretation; e.g., the height of Mount Everest is generally considered “data”, a book on Mount Everest geological characteristics may be considered “information”, and a climber’s guidebook containing practical information on the best way to reach Mount Everest’s peak may be considered “knowledge”. “Information” bears a diversity of meanings that ranges from everyday usage to technical use. This view, however, has also been argued to reverse how data emerges from information, and information from knowledge. Generally speaking, the concept of information is closely related to notions of constraint, communication, control, data, form, instruction, knowledge, meaning, mental stimulus, pattern, perception, and representation. Beynon-Davies uses the concept of a sign to differentiate between data and information; data is a series of symbols, while information occurs when the symbols are used to refer to something.
Before the development of computing devices and machines, people had to manually collect data and impose patterns on it. Since the development of computing devices and machines, these devices can also collect data. In the 2010s, computers are widely used in many fields to collect data and sort or process it, in disciplines ranging from marketing, analysis of social services usage by citizens to scientific research. These patterns in data are seen as information that can be used to enhance knowledge. These patterns may be interpreted as “truth” (though “truth” can be a subjective concept) and may be authorized as aesthetic and ethical criteria in some disciplines or cultures. Events that leave behind perceivable physical or virtual remains can be traced back through data. Marks are no longer considered data once the link between the mark and observation is broken.
Mechanical computing devices are classified according to how they represent data. An analog computer represents a datum as a voltage, distance, position, or other physical quantity. A digital computer represents a piece of data as a sequence of symbols drawn from a fixed alphabet. The most common digital computers use a binary alphabet, that is, an alphabet of two characters typically denoted “0” and “1”. More familiar representations, such as numbers or letters, are then constructed from the binary alphabet. Some special forms of data are distinguished. A computer program is a collection of data, which can be interpreted as instructions. Most computer languages make a distinction between programs and the other data on which programs operate, but in some languages, notably Lisp and similar languages, programs are essentially indistinguishable from other data. It is also useful to distinguish metadata, that is, a description of other data. A similar yet earlier term for metadata is “ancillary data.” The prototypical example of metadata is the library catalog, which is a description of the contents of books.
Minimum information standards are sets of guidelines and formats for reporting data derived by specific high-throughput methods. Their purpose is to ensure the data generated by these methods can be easily verified, analysed and interpreted by the wider scientific community. Ultimately, they facilitate the transfer of data from journal articles (unstructured data) into databases (structured data) in a form that enables data to be mined across multiple data sets. Minimal information standards are available for a vast variety of experiment types including microarray (MIAME
), RNAseq (MINSEQE), metabolomics (MSI) and proteomics (MIAPE).
Minimum information standards typically have two parts. Firstly, there is a set of reporting requirements – typically presented as a table or a checklist. Secondly, there is a data format. Information about an experiment needs to be converted into the appropriate data format for it to be submitted to the relevant database. In the case of MIAME, the data format is provided in spreadsheet format (MAGE-TAB). Some of the communities that maintain minimum information standards also provide tools to help experimental researchers to annotate their data.
The individual minimum information standards are brought by the communities of cross-disciplinary specialists focused on the problematic of the specific method used in experimental biology. The standards then provide specifications what information about the experiments (metadata) is crucial and important to be reported together with the resultant data to make it comprehensive. The need for this standardization is largely driven by the development of high-throughput experimental methods that provide tremendous amounts of data. The development of minimum information standards of different methods is since 2008 being harmonized by “Minimum Information about a Biomedical or Biological Investigation” (MIBBI) project.
MIAPPE is an open, community driven project to harmonize data from plant phenotyping experiments. MIAPPE comprises both a conceptual checklist of metadata required to adequately describe a plant phenotyping experiment.
Published in 2009 these guidelines for the basis of requirements by many journals when submitting QPCR data, sadly they are not adhered to enough.
Minimum Information About a Microarray Experiment (MIAME) describes the Minimum Information About a Microarray Experiment that is needed to enable the interpretation of the results of the experiment unambiguously and potentially to reproduce the experiment and is aimed at facilitating the dissemination of data from microarray experiments. It was published by the FGED Society in 2001 and was the first published minimum information standard for high-throughput experiments in the life sciences.
MIAME contains a number of extensions to cover specific biological domains, including MIAME-env, MIAME-nut and MIAME-tox, covering environmental genomics, nutritional genomics and toxogenomics, respectively.
Electrophysiology is a technology used to study the electrical properties of biological cells and tissues. Electrophysiology typically involves the measurements of voltage change or electric current flow on a wide variety of scales from single ion channel proteins to whole tissues. This document is a single module, as part of the Minimum Information about a Neuroscience investigation (MINI) family of reporting guideline documents, produced by community consultation and continually available for public comment. A MINI module represents the minimum information that should be reported about a dataset to facilitate computational access and analysis to allow a reader to interpret and critically evaluate the processes performed and the conclusions reached, and to support their experimental corroboration. In practice a MINI module comprises a checklist of information that should be provided (for example about the protocols employed) when a data set is described for publication. The full specification of the MINI module can be found here.
A file format is a standard way that information is encoded for storage in a computer file. It specifies how bits are used to encode information in a digital storage medium. File formats may be either proprietary or free.
Some file formats are designed for very particular types of data: PNG files, for example, store bitmapped images using lossless data compression. Other file formats, however, are designed for storage of several different types of data: the Ogg format can act as a container for different types of multimedia including any combination of audio and video, with or without text (such as subtitles), and metadata. A text file can contain any stream of characters, including possible control characters, and is encoded in one of various character encoding schemes. Some file formats, such as HTML, scalable vector graphics, and the source code of computer software are text files with defined syntaxes that allow them to be used for specific purposes.
File formats often have a published specification describing the encoding method and enabling testing of program intended functionality. Not all formats have freely available specification documents, partly because some developers view their specification documents as trade secrets, and partly because other developers never author a formal specification document, letting precedent set by other already existing programs that use the format define the format via how these existing programs use it.
If the developer of a format doesn’t publish free specifications, another developer looking to utilize that kind of file must either reverse engineer the file to find out how to read it or acquire the specification document from the format’s developers for a fee and by signing a non-disclosure agreement. The latter approach is possible only when a formal specification document exists. Both strategies require significant time, money, or both; therefore, file formats with publicly available specifications tend to be supported by more programs.