Monthly Archives: January 2021

Dally Rhythms – 2020.01.31

Tracklist

  • Africanism & Olivia De Lanzac – Chocolate featuring Kawele Multimanwa (Original Mix), A minor, 123 bpm
  • Bahlzack – Air (Original Mix), A minor, 123 bpm
  • Daniel Camarillo – Button (Robert R. Hardy Dark Therapy Remix), A minor, 122 bpm
  • Cristoph – Delusions (Original mix), F minor, 123 bpm
  • Beckwith – End Of Brooklyn (Original Mix), Bb minor, 123 bpm
  • Future Motions, Linton Brown – Stealer (Original Mix), Bb minor, 123 bpm
  • Alex Aark, Meron – Along With The Stars (Original Mix), F minor, 122 bpm
  • Hak – Disco Fantasy (Jojo Rose & Panos Gerontakis Remix), C minor, 120 bpm 
  • Chab & JD Davis – Lonely (Simon & Shaker Class Remix), C minor, 124 bpm 
  • Andres Moris And Pedro Belardo – Exile (Original Mix), Eb minor, 122 bpm
  • Kury Lenko – Autumn Leaves Again (Original Mix), C minor, 124 bpm 
  • Eli & Fur – Night Blooming Jasmine (Rodriguez Jr Remix), F minor, 120 bpm

Available for download in the archives.

SUYF! – Playlist 1/31/21

Songs inspired by deception, delusion and double standards. R.I.P. Cicely.

  • ARCHIVE
  • Love Will Tear Us Apart – Suzanna and the Magical Orchestra
  • My Right Versus Yours – The New Pornographers
  • Slave to the Rhythm = Grace Jones
  • Slave Song – Sade
  • Slave Driver – Bob Marley & The Wailers
  • Slave – The Rolling Stones
  • How The West Was Won And Where It Got Us – R.E.M.
  • The Boy in the Bubble – Paul Simon
  • Political World – Bob Dylan
  • Sword of Damocles (Externally) – Lou Reed
  • Language is a Virus – Laurie Anderson
  • Big Science – Laurie Anderson
  • Natural Science – Rush
  • The Sounds of Science – Beastie Boys
  • Science is Real – They Might Be Giants
  • The Sexual Loneliness of Jesus Christ – Jackie Leven
  • Love Minus Zero / No Limit – Bob Dylan
  • There She Goes, My Beautiful World – Nick Cave & The Bad Seeds
  • Innocent When You Dream (Barroom) – Tom Waits
  • I Often Dream of Trains – Robyn Hitchcock
  • Ship of Fools – John Cale
  • Medicine Show – Big Audio Dynamite
  • Lose This Skin – The Clash
  • When I Get To The Border – Richard & Linda Thompson
  • We Take Care of Our Own – Bruce Springsteen

The Server Room Show – Episode 62 – Solarwinds and its aftermath

What is Solarwinds

Solarwinds is a multinational company with over 3000 employees and 300 000 clients worldwide. A major IT firm that provides software for entities ranging from Fortune 500 companies to the US Government.

Solarwinds main product the Orion platform is a powerful, scalable infrastructure monitoring and management platform designed to simplify IT administration for on-premises, hybrid, and software as a service (SaaS) environments in a single pane of glass

https://cdn.vidyard.com/thumbnails/4866758/vgvdv4LwvwYkE9RgpTIXfQ.png
Example Dashboard of SolarWinds Orion platform single pane of glass


The Solarwinds hack and why it is such a big deal ( or is it?)
SUNBURST and SUPERNOVA attacks

SUNBURST

They call it the most serios cyber attack agains an enterprise software giant ever

Reuters first reported that SolarWinds was the subject of a massive cybersecurity attack that spread to the company’s clients.

The breach went undetected for months and could have exposed data in the highest reaches of government including the US military and the White House.

As always US officials thinks it was the Russians behind it
( are not they behind everything?? but do we have enough russians to be behind everything? 🙂 )

Whoever was/were behind this hack were able to use it to spy on private companies like the elite cybersecurity firm FireEye and the US Government including the Department of Homeland Security and Treasury Department.

Earlier in 2020 hackers secretly broke into Texas-based SolarWind’s systems and added malicious code into the company’s software system. The system, called “Orion,” is widely used by companies to manage IT resources. Solarwinds has 33,000 customers that use Orion

Most software providers regularly send out updates to their systems, whether it’s fixing a bug or adding new features. SolarWinds is no exception. Beginning as early as March, SolarWinds unwittingly sent out software updates to its customers that included the hacked code. 

The code created a backdoor to customer’s information technology systems, which hackers then used to install even more malware that helped them spy on companies and organizations.

The attack used a backdoor in a SolarWinds library; when an update to SolarWinds occurred the malicious attack would go unnoticed due to the trusted certificate. In November 2019, a security researcher notified SolarWinds that their FTP server had a weak password of “solarwinds123”, warning that “any hacker could upload malicious [files]” that would then be distributed to SolarWinds customers.

The New York Times reported SolarWinds did not employ a chief information security officer and that employee passwords had been posted on GitHub in 2019 other sources however estimates that the leak through githubs public repo of Solarwinds was going on since 2018 (Leaked ftp credentials & weak ftp password) <<< Security researcher Vinoth Kumar alerted the company in 2019 about the ftp password leak and that anyone could acces SolarWinds update server by using the password “solarwinds123”

On December 15, 2020, SolarWinds reported the breach to the Securities and Exchange Commission. However, SolarWinds continued to distribute malware-infected updates, and did not immediately revoke the compromised digital certificate used to sign them.

On December 16, 2020, German IT news portal Heise.de reported that SolarWinds had for some time been encouraging customers to disable anti-malware tools before installing SolarWinds products.

On December 17, 2020, SolarWinds said they would revoke the compromised certificates by December 21, 2020.

SUPERNOVA

On December 19, 2020, Microsoft said that its investigations into supply chain attacks at SolarWinds had found evidence of an attempted supply chain attack distinct from the attack in which SUNBURST malware was inserted into Orion binaries (see previous section). This second attack has been dubbed SUPERNOVA 

Security researchers from Palo Alto Networks said the SUPERNOVA malware was implemented stealthily. SUPERNOVA comprises a very small number of changes to the Orion source code, implementing a web shell that acts as a remote access tool. The shell is assembled in-memory during SUPERNOVA execution, thus minimizing its forensic footprint.

Unlike SUNBURST, SUPERNOVA does not possess a digital signature. This is among the reasons why it is thought to have originated with a different group than the one responsible for SUNBURST.

Insider trading investigation

SolarWinds’s share price fell 25% within days of the SUNBURST breach becoming public knowledge and 40% within a week. Insiders at the company had sold approximately $280 million in stock shortly before this became publicly known which was months after the attack had started. A spokesperson said that those who sold the stock had not been aware of the breach at the time.

Just the good opportunity for some other companies

Microsoft (Azure) and a Spanish startup called Artica ( sums about 40 workers and around 400 clients) which has its own product in the systems monitoring market. Many of Solarwinds client are looking for alternatives and they want a way out and they are looking at alternative offers like Artica’s monitoring solutions ( a customized Pandora FMS) or trying to avoid a similar issue to happen by moving their infrastructure to the cloud like Microsoft’s Azure.

Probably it offers a great opportunity for many of Solarwinds competitors including OpenNMS who’s founder and CEO Tarus Balog will have a sit down with me and chat amongst other things of this particular event as well on the next episode 63 of The Server Room Show

Examples of BAD Behaviour in the Solarwinds story

  • Weak Passwords Anywhere ( Solarwinds FTP password “solarwinds123” )
  • Telling Your users not to use anti-malware ( Making Solarwinds responsible cause They have adviced so their users)
  • Not listening and following up when security researchers find and tell your company about a vulnerability cause you are too busy and arrogant perhaps?
  • Not revoking compromised certificates inmediatelly
  • Not employing a CISO – Chief Information Security Office in a company of a size and customer base like Solarwinds ( reaching to Fortune 500 and Government entities)

Things to keep in mind

  • Anything can be compromised and nothing is safe as 100%
  • Many times bad security practices help hackers to gain access easier than we would imagine
  • The stuck Your head in the sand approach do not work ever
  • Do not advise something to your clients you might not be doing yourself
Links


https://www.businessinsider.com/solarwinds-hack-explained-government-agencies-cyber-security-2020-12
https://www.marketwatch.com/story/microsoft-earnings-the-solarwinds-hack-may-be-a-good-thing-for-azure-11611352122
https://www.newyorker.com/news/daily-comment/after-the-solarwinds-hack-we-have-no-idea-what-cyber-dangers-we-face
https://www.expansion.com/economia-digital/companias/2018/09/21/5ba0b7cbe2704e11b78b457e.html
https://artica.es/en/
https://pandorafms.com/
https://www.cvedetails.com/product-list/vendor_id-1305/Solarwinds.html
https://savebreach.com/solarwinds-exposed-ftp-credentials-back-in-2018-says-security-researcher-vinoth/
https://www.reuters.com/article/global-cyber-solarwinds/hackers-at-center-of-sprawling-spy-campaign-turned-solarwinds-dominance-against-it-idUSKBN28P2N8

FroggyMe’s Fantastic Fantasy – playlist for Jan 24th, 2021

Meco – Other Galactic Funk – Star Wars and Other Galactic Funk
Lester Bowie – The Great Pretender – The Great Pretender (ECM)
Lester Bowie – It’s Howdie Doodie Time – The Great Pretender (ECM)
Lester Bowie – Doom? – The Great Pretender (ECM)
J.J. Johnson – The Seventh Son
J.J. Johnson – Stir Fry – Let’s Hang Out
Raul de Souza – At Will – Sweet Lucy

Incognito – Black Rain – No Time Like The Future
McCoy Tyner – Beyond The Sun – Fly With The Wind

Dally Rhythms – 2021.01.24

Tracklist

  • Angga – Cosmic Structure, D minor, 122 bpm
  • Brian Cid – Eternal Sunrise (Original Mix), D minor, 121 bpm
  • Animal Picnic & Ran Salman – Mind Maze (Original Mix), D minor, 121 bpm
  • Agraba, Escenda – Do it Again! (Original Mix), A minor, 119 bpm
  • Benn Finn – Soul Searching (Mirco Niemeier Remix), A minor, 120 bpm
  • Bedouin Soundclash – Brutal Hearts (FlicFlac Edit), A minor, 120 bpm
  • Bes & Meret – Analog Bubbles, Ab minor, 120 bpm
  • Alessio Cala’ – 1996 (Original Mix), Ab minor, 125 bpm
  • Enrico Sangiuliano – X-Pollination (Original Mix), Ab minor, 126 bpm
  • Drvg Cvltvre – Langorous (Mary Velo Remix), A minor, 127 bpm
  • Hans Bouffmyhre – Midsummer Beatdown, A minor, 128 bpm
  • Subjected – My Unclear Expression, A minor, 128 bpm

Available for download in the archives.

SUYF! – Playlist 1/24/21

Songs inspired by relief, celebration and uncertainty. R.I.P. Hank, Larry and Gregory.

  • ARCHIVE
  • Requiem – King Crimson
  • Jump Around – House of Pain
  • Hits from the Bong – Cypress Hill
  • The Winner Takes It All – ABBA
  • Since You’re Gone – The Cars
  • Alabama Song (Whisky Bar) – The Doors
  • Ain’t Talkin’ – Bob Dylan
  • When the Shit Goes Down – Cypress Hill
  • Body Count – Ice-T
  • Armageddon Man – Black Flag
  • Account for What? – Black Flag
  • Sabotage – Beastie Boys
  • Don’t Believe the Hype – Public Enemy
  • Follow The Leader – Eric B. & Rakim
  • Beercan – Beck
  • Fuck Em – Geto Boys
  • Jesus Built My Hotrod – Ministry
  • Just Another Victim – Helmet, House of Pain
  • Detachable Penis – King Missile
  • I Love You, Mary Jane – Sonic Youth, Cypress Hill
  • Been Caught Stealing – Jane’s Addiction
  • Party at Ground Zero – Fishbone
  • United States of Whatever – Liam Lynch
  • Dad I’m in Jail – Don Was
  • Beer, Steers & Queers – Revolting Cocks
  • Are You Drinkin’ With Me Jesus – Jello Biafra, Mojo Nixon
  • Let’s Get Fucked Up – The Cramps

FroggyMe’s Fantastic Fantasy – playlist for January 17th, 2021

Weldon Irvine – Music is the Key
Alphonse Mouzon – Do I Have To? – By All Means
Incognito – Colibri (Talkin’ Loud)
Antonio Carlos Jobim – Wave – Wave (CTI)
Perception – Serious Love (Talkin’ Loud)
Antonio Carlos Jobim – Batidinha – Wave (CTI)
JTQ – 3 Mile Island – In the Hand of the Inevitable (Acid Jazz)
(The New) Tony Williams Lifetime – Proto-Cosmos – Believe It


Pleasure – Future Now – Future Now


(The New) Tony Williams Lifetime – Snake Oil – Believe It
EW&F – Sun Goddess (Ramsey Lewis) – Plugged In And Live

Dally Rhythms – 2021.01.17

Tracklist

  • Ghebro – Burning In My Soul (Original Mix), C minor, 120 bpm 
  • Ben-E & Falki – Place That You Feel (Original Mix), C minor, 120 bpm 
  • Max Manie – Sunday (Klangtherapeuten Looking For Summer Remix), Eb minor, 124 bpm
  • Charming Horses – Follow featuring Jona Bird (Lizot Remix), C minor, 120 bpm 
  • HollyHenry – Seven Nation Army (Teemid Cover), C minor, 120 bpm 
  • Aytac Kart – Phobias and Fears (Original mix), G minor, 120 bpm 
  • A.Shine & Alex Project – Believe (Andro V & Andrew Krasnov Deep House Mix), C minor, 122 bpm
  • David Devilla, Elisabeth Aivar – Dont Walk (Alvaro Smart Remix), C minor, 122 bpm 
  • Eelke Kleijn – Mistakes I’ve Made (Original Mix), G minor, 120 bpm 
  • Face Off – Let It All Down (Vijay & Sofia Zlatko Remix), G minor, 120 bpm 
  • Choir Of Young Believers – Hollow Talk (Lane8 & Jody Wisternoff & James Grant Remix), D minor, 120 bpm 
  • Jan Blomqvist – Dancing People Are Never Wrong featuring The Bianca Story (Budakid Extended Remix), D minor, 121 bpm
  • Catching Flies – Satisfied (Ambient Reprise), A minor, 122 bpm
  • Catching Flies – Satisfied (Marsh Extended Mix), A minor, 122 bpm

Available for download in the archives.

SUYF! – Playlist 1/17/21

Songs inspired by comeuppance. R.I.P. Sylvain and Marsha. Oh Phil.

1968
  • https://archives.anonradio.net/202101171700_djrob.mp3
  • No Surrender (Live)- Bruce Springsteen & The E Street Band
  • Confrontation Music – The Greg Kihn Band
  • You’ve Got Another Thing Coming – Judas Priest
  • I Predict a Riot – Kaiser Chiefs
  • White Riot – The Clash
  • Wave of Mutilation – Pixies
  • Guilty of Being White – Minor Threat
  • I Wanna Destroy You – The Soft Boys
  • Destroyer – The Kinks
  • Take Me to the Riot – Stars
  • Make America Great Again – Pussy Riot
  • Teen Age Riot – Sonic Youth
  • Riot Van – Arctic Monkeys
  • Caught by the Fuzz – Supergrass
  • Take the Skinheads Bowling – Camper Van Beethoven
  • Mind Riot – Soundgarden
  • Panic – The Smiths
  • Clampdown – The Clash
  • Mierda De Ciudad – Kortatu
  • Makes No Sense At All – Hüsker Dü
  • It’s the End of the World As We Know It – R.E.M.
  • Uncontrollable Urge – DEVO
  • Secret Agent Man – DEVO
  • You Can Run – A Flock of Seagulls
  • Shots – Neil Young
  • Surrender – Elvis Presley
  • Surrender – The J. Geils Band
  • Surrender – Cheap Trick
  • Surrender – Suicide
  • Personality Crisis (Live) – New York Dolls
  • Human Being (Live) – New York Dolls

The Server Room Show – Episode 60 – Database

Prologue

A database is an organized collection of data, generally stored and accessed electronically from a computer system. Where databases are more complex they are often developed using formal design and modeling techniques which i will talk about a bit later.

An SQL select statement and its result.

The database management system (DBMS) is the software that interacts with end users, applications, and the database itself to capture and analyze the data. The DBMS software additionally encompasses the core facilities provided to administer the database. The sum total of the database, the DBMS and the associated applications can be referred to as a “database system”. Often the term “database” is also used to loosely refer to any of the DBMS, the database system or an application associated with the database.

Computer scientists may classify database-management systems according to the database models that they support. Relational databases became dominant in the 1980s. These model data as rows and columns in a series of tables, and the vast majority use SQL for writing and querying data. In the 2000s, non-relational databases became popular, referred to as NoSQL because they use different query languages.

Terminology and Overview

Formally, a “database” refers to a set of related data and the way it is organized. Access to this data is usually provided by a “database management system” (DBMS) consisting of an integrated set of computer software that allows users to interact with one or more databases and provides access to all of the data contained in the database (although restrictions may exist that limit access to particular data). The DBMS provides various functions that allow entry, storage and retrieval of large quantities of information and provides ways to manage how that information is organized.

Because of the close relationship between them, the term “database” is often used casually to refer to both a database and the DBMS used to manipulate it.

Outside the world of professional information technology, the term database is often used to refer to any collection of related data (such as a spreadsheet or a card index) as size and usage requirements typically necessitate use of a database management system.

Existing DBMSs provide various functions that allow management of a database and its data which can be classified into four main functional groups:

  • Data definition – Creation, modification and removal of definitions that define the organization of the data.
  • Update – Insertion, modification, and deletion of the actual data.
  • Retrieval – Providing information in a form directly usable or for further processing by other applications. The retrieved data may be made available in a form basically the same as it is stored in the database or in a new form obtained by altering or combining existing data from the database.
  • Administration – Registering and monitoring users, enforcing data security, monitoring performance, maintaining data integrity, dealing with concurrency control, and recovering information that has been corrupted by some event such as an unexpected system failure.

Both a database and its DBMS conform to the principles of a particular database model. Database system” refers collectively to the database model, database management system, and database.

Physically, database servers are dedicated computers that hold the actual databases and run only the DBMS and related software. Database servers are usually multiprocessor computers, with generous memory and RAID disk arrays used for stable storage. Hardware database accelerators, connected to one or more servers via a high-speed channel, are also used in large volume transaction processing environments. DBMSs are found at the heart of most database applications. DBMSs may be built around a custom multitasking kernel with built-in networking support, but modern DBMSs typically rely on a standard operating system to provide these functions.

Since DBMSs comprise a significant market, computer and storage vendors often take into account DBMS requirements in their own development plans.

Databases and DBMSs can be categorized according to the database model(s) that they support (such as relational or XML), the type(s) of computer they run on (from a server cluster to a mobile phone), the query language(s) used to access the database (such as SQL or XQuery), and their internal engineering, which affects performance, scalability, resilience, and security.

History

The sizes, capabilities, and performance of databases and their respective DBMSs have grown in orders of magnitude. These performance increases were enabled by the technology progress in the areas of processors, computer memory, computer storage, and computer networks. The concept of a database was made possible by the emergence of direct access storage media such as magnetic disks, which became widely available in the mid 1960s; earlier systems relied on sequential storage of data on magnetic tape. The subsequent development of database technology can be divided into three eras based on data model or structure:

  • navigational
  • SQL/relational
  • post-relational

The two main early navigational data models were the hierarchical model and the CODASYL model (network model). These were characterized by the use of pointers (often physical disk addresses) to follow relationships from one record to another.

The relational model, first proposed in 1970 by Edgar F. Codd, departed from this tradition by insisting that applications should search for data by content, rather than by following links. The relational model employs sets of ledger-style tables, each used for a different type of entity. Only in the mid-1980s did computing hardware become powerful enough to allow the wide deployment of relational systems (DBMSs plus applications). By the early 1990s, however, relational systems dominated in all large-scale data processing applications, and as of 2018 they remain dominant: IBM DB2, Oracle, MySQL, and Microsoft SQL Server are the most searched DBMS. The dominant database language, standardised SQL for the relational model, has influenced database languages for other data models.

Object databases were developed in the 1980s to overcome the inconvenience of object-relational impedance mismatch, which led to the coining of the term “post-relational” and also the development of hybrid object-relational databases.

The next generation of post-relational databases in the late 2000s became known as NoSQL databases, introducing fast key-value stores and document-oriented databases. A competing “next generation” known as NewSQL databases attempted new implementations that retained the relational/SQL model while aiming to match the high performance of NoSQL compared to commercially available relational DBMSs.

1960s, navigational DBMS

The introduction of the term database coincided with the availability of direct-access storage (disks and drums) from the mid-1960s onwards. The term represented a contrast with the tape-based systems of the past, allowing shared interactive use rather than daily batch processing. The Oxford English Dictionary cites a 1962 report by the System Development Corporation of California as the first to use the term “data-base” in a specific technical sense.[10]

As computers grew in speed and capability, a number of general-purpose database systems emerged; by the mid-1960s a number of such systems had come into commercial use. Interest in a standard began to grow, and Charles Bachman, author of one such product, the Integrated Data Store (IDS), founded the Database Task Group within CODASYL, the group responsible for the creation and standardization of COBOL. In 1971, the Database Task Group delivered their standard, which generally became known as the CODASYL approach, and soon a number of commercial products based on this approach entered the market.

The CODASYL approach offered applications the ability to navigate around a linked data set which was formed into a large network. Applications could find records by one of three methods:

  1. Use of a primary key (known as a CALC key, typically implemented by hashing)
  2. Navigating relationships (called sets) from one record to another
  3. Scanning all the records in a sequential order

Later systems added B-trees to provide alternate access paths. Many CODASYL databases also added a declarative query language for end users (as distinct from the navigational API). However CODASYL databases were complex and required significant training and effort to produce useful applications.

IBM also had their own DBMS in 1966, known as Information Management System (IMS). IMS was a development of software written for the Apollo program on the System/360. IMS was generally similar in concept to CODASYL, but used a strict hierarchy for its model of data navigation instead of CODASYL’s network model. Both concepts later became known as navigational databases due to the way data was accessed: the term was popularized by Bachman’s 1973 Turing Award presentation The Programmer as Navigator. IMS is classified by IBM as a hierarchical database. IDMS and Cincom SystemsTOTAL database are classified as network databases. IMS remains in use as of today its last stable release has been on 2017

1970s, relational DBMS

Edgar F. Codd worked at IBM in San Jose, California, in one of their offshoot offices that was primarily involved in the development of hard disk systems. He was unhappy with the navigational model of the CODASYL approach, notably the lack of a “search” facility. In 1970, he wrote a number of papers that outlined a new approach to database construction that eventually culminated in the groundbreaking A Relational Model of Data for Large Shared Data Banks.[12]

In this paper, he described a new system for storing and working with large databases. Instead of records being stored in some sort of linked list of free-form records as in CODASYL, Codd’s idea was to organise the data as a number of “tables“, each table being used for a different type of entity. Each table would contain a fixed number of columns containing the attributes of the entity. One or more columns of each table were designated as a primary key by which the rows of the table could be uniquely identified; cross-references between tables always used these primary keys, rather than disk addresses, and queries would join tables based on these key relationships, using a set of operations based on the mathematical system of relational calculus (from which the model takes its name). Splitting the data into a set of normalized tables (or relations) aimed to ensure that each “fact” was only stored once, thus simplifying update operations. Virtual tables called views could present the data in different ways for different users, but views could not be directly updated.

Codd used mathematical terms to define the model: relations, tuples, and domains rather than tables, rows, and columns. The terminology that is now familiar came from early implementations. Codd would later criticize the tendency for practical implementations to depart from the mathematical foundations on which the model was based.

In the relational model, records are “linked” using virtual keys not stored in the database but defined as needed between the data contained in the records.

The use of primary keys (user-oriented identifiers) to represent cross-table relationships, rather than disk addresses, had two primary motivations. From an engineering perspective, it enabled tables to be relocated and resized without expensive database reorganization. But Codd was more interested in the difference in semantics: the use of explicit identifiers made it easier to define update operations with clean mathematical definitions, and it also enabled query operations to be defined in terms of the established discipline of first-order predicate calculus; because these operations have clean mathematical properties, it becomes possible to rewrite queries in provably correct ways, which is the basis of query optimization. There is no loss of expressiveness compared with the hierarchic or network models, though the connections between tables are no longer so explicit.

In the hierarchic and network models, records were allowed to have a complex internal structure. For example, the salary history of an employee might be represented as a “repeating group” within the employee record. In the relational model, the process of normalization led to such internal structures being replaced by data held in multiple tables, connected only by logical keys.

For instance, a common use of a database system is to track information about users, their name, login information, various addresses and phone numbers. In the navigational approach, all of this data would be placed in a single variable-length record. In the relational approach, the data would be normalized into a user table, an address table and a phone number table (for instance). Records would be created in these optional tables only if the address or phone numbers were actually provided.

As well as identifying rows/records using logical identifiers rather than disk addresses, Codd changed the way in which applications assembled data from multiple records. Rather than requiring applications to gather data one record at a time by navigating the links, they would use a declarative query language that expressed what data was required, rather than the access path by which it should be found. Finding an efficient access path to the data became the responsibility of the database management system, rather than the application programmer. This process, called query optimization, depended on the fact that queries were expressed in terms of mathematical logic.

Codd’s paper was picked up by two people at Berkeley, Eugene Wong and Michael Stonebraker. They started a project known as INGRES using funding that had already been allocated for a geographical database project and student programmers to produce code. Beginning in 1973, INGRES delivered its first test products which were generally ready for widespread use in 1979. INGRES was similar to System R in a number of ways, including the use of a “language” for data access, known as QUEL. Over time, INGRES moved to the emerging SQL standard.

IBM itself did one test implementation of the relational model, PRTV, and a production one, Business System 12, both now discontinued. Honeywell wrote MRDS for Multics, and now there are two new implementations: Alphora Dataphor and Rel. Most other DBMS implementations usually called relational are actually SQL DBMSs.

In 1970, the University of Michigan began development of the MICRO Information Management System[13] based on D.L. Childs‘ Set-Theoretic Data model.[14][15][16] MICRO was used to manage very large data sets by the US Department of Labor, the U.S. Environmental Protection Agency, and researchers from the University of Alberta, the University of Michigan, and Wayne State University. It ran on IBM mainframe computers using the Michigan Terminal System.[17] The system remained in production until 1998.

Late 1970s, SQL DBMS

IBM started working on a prototype system loosely based on Codd’s concepts as System R in the early 1970s. The first version was ready in 1974/5, and work then started on multi-table systems in which the data could be split so that all of the data for a record (some of which is optional) did not have to be stored in a single large “chunk”. Subsequent multi-user versions were tested by customers in 1978 and 1979, by which time a standardized query language – SQL[citation needed] – had been added. Codd’s ideas were establishing themselves as both workable and superior to CODASYL, pushing IBM to develop a true production version of System R, known as SQL/DS, and, later, Database 2 (DB2).

Larry Ellison‘s Oracle Database (or more simply, Oracle) started from a different chain, based on IBM’s papers on System R. Though Oracle V1 implementations were completed in 1978, it wasn’t until Oracle Version 2 when Ellison beat IBM to market in 1979.[18]

Stonebraker went on to apply the lessons from INGRES to develop a new database, Postgres, which is now known as PostgreSQL. PostgreSQL is often used for global mission-critical applications (the .org and .info domain name registries use it as their primary data store, as do many large companies and financial institutions).

In Sweden, Codd’s paper was also read and Mimer SQL was developed from the mid-1970s at Uppsala University. In 1984, this project was consolidated into an independent enterprise.

Another data model, the entity–relationship model, emerged in 1976 and gained popularity for database design as it emphasized a more familiar description than the earlier relational model. Later on, entity–relationship constructs were retrofitted as a data modeling construct for the relational model, and the difference between the two have become irrelevant.

1980s, on the desktop

The 1980s ushered in the age of desktop computing. The new computers empowered their users with spreadsheets like Lotus 1-2-3 and database software like dBASE. The dBASE product was lightweight and easy for any computer user to understand out of the box. C. Wayne Ratliff, the creator of dBASE, stated: “dBASE was different from programs like BASIC, C, FORTRAN, and COBOL in that a lot of the dirty work had already been done. The data manipulation is done by dBASE instead of by the user, so the user can concentrate on what he is doing, rather than having to mess with the dirty details of opening, reading, and closing files, and managing space allocation.”[19] dBASE was one of the top selling software titles in the 1980s and early 1990s.

1990s, object-oriented

The 1990s, along with a rise in object-oriented programming, saw a growth in how data in various databases were handled. Programmers and designers began to treat the data in their databases as objects. That is to say that if a person’s data were in a database, that person’s attributes, such as their address, phone number, and age, were now considered to belong to that person instead of being extraneous data. This allows for relations between data to be relations to objects and their attributes and not to individual fields.[20] The term “object-relational impedance mismatch” described the inconvenience of translating between programmed objects and database tables. Object databases and object-relational databases attempt to solve this problem by providing an object-oriented language (sometimes as extensions to SQL) that programmers can use as alternative to purely relational SQL. On the programming side, libraries known as object-relational mappings (ORMs) attempt to solve the same problem.

2000s, NoSQL and NewSQL

XML databases are a type of structured document-oriented database that allows querying based on XML document attributes. XML databases are mostly used in applications where the data is conveniently viewed as a collection of documents, with a structure that can vary from the very flexible to the highly rigid: examples include scientific articles, patents, tax filings, and personnel records.

NoSQL databases are often very fast, do not require fixed table schemas, avoid join operations by storing denormalized data, and are designed to scale horizontally.

In recent years, there has been a strong demand for massively distributed databases with high partition tolerance, but according to the CAP theorem it is impossible for a distributed system to simultaneously provide consistency, availability, and partition tolerance guarantees. A distributed system can satisfy any two of these guarantees at the same time, but not all three. For that reason, many NoSQL databases are using what is called eventual consistency to provide both availability and partition tolerance guarantees with a reduced level of data consistency.

NewSQL is a class of modern relational databases that aims to provide the same scalable performance of NoSQL systems for online transaction processing (read-write) workloads while still using SQL and maintaining the ACID guarantees of a traditional database system.

Links

https://en.wikipedia.org/wiki/IBM_Information_Management_System