How is a SQL server better than a set of sorted CSV file.
Discussion
"The initial simplicity of CSV files quickly becomes a liability as data complexity and usage requirements grow."
Yes.
I prefer to work with the csv. Also easier for everyone at every skill set to work with when collaborating.
This was a hot take I didn't expect anyone to actually agree with 🤝
This is how every business ends up with an untenable set of excel docs with spaghetti code macros
You have to think: why is Excel the best consumer-friendly database management system we have? Somehow it's the Schelling point in a world where dedicated DBMS's exist.
My answer is that DBMS's are developer facing solutions, and people aren't going to hire a developer over just learning some VBA. And this is somehow especially true in fields with countless data management solutions built specially for them, yet Excel is still the glue that holds it all together.
I used to agree until I stated working directly with clients. Many of them especially small clients, don’t have people who can use SQL. So they need data in a simple delineated file. While SQL is more efficient, it results in data that is unusable by many. Everyone knows excel or a similar spreadsheet. For large companies or companies with a dedicated data management person/department SQL makes more sense.
How is a data management system better than a file format?
Ever tried loading 30GB CSVs into ram?
Point is with SQL you have 40 years of very smart engineers, figuring out how to load and save data and deal with memory and disk complexities very fucking well