10 Reasons Databases Suck

I'm glad to see that you brought this up Joel, perhaps Christopher or someone can elaborate.

How does OpenEdit work without a database?

What are the advantages/disadvantages of not using a database?

Your post states that databases have many limitations but doesn't explain why no database makes OpenEdit a better or more reliable ECM.

Typically when people hear of no-database CMS solutions, they think of very small or hobbyist CMS solutions, however;  OpenEdit is used successfully in global enterprise environments for many corporate marketing departments and firms. Can you address this?

There are lots of reasons why a database is not an absolute requirement for complex web applications.  But the best approach is to abstract out a few things and give yourself a persistence layer, leaving people with options.  Ian demonstrated OpenEdit's persistence layer to me the other day.  The fact is that in analyzing customer requirements there hasn't been a need for a relational database under the covers yet.  But someday a legacy system might be there which is all about SQL queries... and OpenEdit is ready to heed that call.

Filesystems excel at all the points you have mentioned, they existed before databases, shoot at the same target audience (at the beginning, at least), come with your default O.S. and nobody uses them for serious work.

Calling the "emperor - no clothes" card is funny, but in this case it seems like relational databases got where they are for actual merit.

Cool. Next time my company wants to hire a DBA I'll ask them to respond to each of those. I think even a junior DBA could solve most of those problems without too much effort.

alter table myTable add myNewColumn varchar(50)

There, new field, took ten seconds.

Most of the other items were about equally valid, at least for the full-featured commercial database I'm most familiar with.

Exactly! That's the whole point. With a file based system you don't need a DBA. You can access the data just the same as you can using Windows explorer. That's the whole point. You get the benefits without the hassle. Last I checked a DBA, even a junior one, costs more than the web application it serves.

So the real question is.. why work to overcome problems that don't need to exist within your web application in the first place?

In OpenEdit we store data all kinds of ways; XML, Hibernate, Lucene, Cumulus, CSV, JDBC, DB4O. What we have found is the most tedious and time consuming way is relational databases. XML is always the easiest.

It seems that in the 80's and 90's relational databases helped a business pull together their data into one place. You could cross reference everything within one company on one server. But now we are building web applications. There are now MANY servers. Each business has it's own web sites and there are no fixed relationships and no data integrity among web sites. If a record is missing then just show an error. If an error occurs on a web form we don't rollback. We let them try again. We don't want the  transaction canceled from Amazon.com if we type in a bad credit card number.

We sometimes work on a large database applications for a fortune 50 companies. We spent 60% of our time doing import/export from mainframe/SQL data into lots of DB tables. We have to get everything approved with the DBA. Have to have transactions, hundreds of fields, 3rd normal form, lots of tables everyplace. But the customers could care less about our 3rd normal form. What real customers really always want to know is how does the user Interface work? How friendly is it? Can it tie into my other systems on other servers?

We should work on the user interface and not pay so much attention to the data. Most data in a database in temporary anyways. After two years most people want to archive off the old data from the database. Databases are old news.
 

To elaborate about adding fields. To add a simple field to a UI screen is hard. First you have to get approval from the DBA and add the column in your test database. Now your code will not match your production server. So your application cannot be tested in production without a new schema change to production. You also need to make sure all your SQL, reports, schema creation scripts know about this new column. You need to make sure your programming Objects and data brokering settings are aware of the column. Then you add the field to the screen.

Also, Database columns are all or nothing. Say you have a products table. Once you add/remove a column to that table then all the products have that field, no exceptions. So you can't have a condition that applies to some of the products but not the rest. You can't say the 3000 products entered in 2007 have these columns but the 5000 products from this year don't.

In summary: column changes cause compatibility problems, slow down changes and do not allow flexibility within a single data type.

Makes a little more sense, but...needing DBA approval is a corporate bureaucracy problem, not a technology problem. Where I work, developers have the power.

Code mismatching on test vs. production works the same way with all my non-database code, too.

My reports don't have to know about the new field, unless somebody actually wants to see the field on that report. Most reports don't show all fields in the database.

Building a creation script is a button push. There are even tools that compare two datatabases and give you a script for the differences.

New fields can be nullable, and if you want to require/prohibit nulls for a given year you could use a trigger. Or instead of nulls you could use a side table.

Adding the field to a UI screen is a UI coding problem, not a database problem. Other data storage systems will still require that. At least with sql, you can query for the database structure, which means people can and do build tools that automatically generate your UI, as long as the generator fits your requirements. Ruby on Rails is one popular example.

I have a suspicion that developers who hate relational databases had the misfortune to work at a corporation with control freak bureaucracies. At small companies like mine, making database changes is the least of our worries, and doesn't get in our way at all. Generally it's the easiest part.

Sigh had to respond.  Having unrelated XML files is not how corporations deal with data across the enterprise,  latest push is SOA/WOA which relies on orchestration to pull data together for analysis.  Other means were cobra, file interchange etc. 

"Say you have a products table. Once you add/remove a column to that table then all the products have that field, no exceptions. So you can't have a condition that applies to some of the products but not the rest. You can't say the 3000 products entered in 2007 have these columns but the 5000 products from this year don't."

Not true, simply add a related table with a primary key pointing to the product key,  if it were something like custom user maintained properties, use a tuple table.

That said, I think XML is a good fit for Openedit.  Just need an easier way to maintain them :)

XML is a great way of describing custom data, as the store functionality can rely heavily on custom properties, XML is a great fit.

XML is not meant to replace DB's.  They both have their place.

"I'll leave you with this. Could you organize over 11 000 photos from 15 photographers taken over 4 days of competition without a database?"

yes, and I could do it without XML as well.  I could use EDI, or a flat file, CSV, TSV, MS Word, straight HTML with attribute tagging etc etc..