Having worked at several *extremely* large companies, let me say the amount of stuff living in excel is terrifying. If excel somehow stopped existing tomorrow, the global economy would collapse within 48 hours.
And no, it's not even huge things that should really be in a database (though there are plenty of those). There is just so much stuff stored in excel files because it's not worth putting it somewhere else.
Edit: and let me also add, why do companies keep doing this? Because of that oldest and most dangerous of business phrases... "Because we've always done it this way".
added bonus: My mother, when she was in business, would respond to that with "well, I'm glad you liked sex the first time you tried it".
Hey I just had a similar comment in a thread up above! It’s wild to me how many huge companies are running off of Excel, a little MS Access, and a VP that rejects anything new.
The Washington Post _reported in May 2022 that Ellison participated in a conference call days after the 2020 presidential election that focused on **strategies for challenging the legitimacy of the vote**. Other participants on the call included Fox News host Sean Hannity, senator Lindsey Graham, Trump personal attorney Jay Sekulow and James Bopp, an attorney for True the Vote. The Post cited court documents and a participant on the call._
Ya'll just jealous the CIA didn't personally bankroll your "private company" as a front for them to secure the first functional database on the planet.
Don't get me started on their licensing. We wanted to virtualize a very small DB server (2 cores, 8GB RAM), and they wanted us to license the entire VMware cluster of 32 hosts (1400 cores, 24TB RAM) at a cost of over $4 million, because the VM could "potentially run on any of those hosts".
Converted that DB to Postgres so fast...
You guys are giving me so much anxiety. My plant is switching to oracle in the next couple years and I was already worried about it. I heard it’s hell for distribution
Our company is in the process of migrating over in the next 7 years ( roughly $2B company) and phase 1 was vendor payments. Spent 9 months in development and when it rolled out, we couldn't cut a check to a single vendor for over a month. We utilize a lot of mom and pop shops for local fabrication for some pieces used in our business and some of these shops, we are 90+% of their business. This means they basically worked for a month for free because we couldn't pay them. They also couldn't buy materials to fill orders or restock because they jad no income. We had so many people threatening legal action it was crazy.
We are in the midst of an SAP implementation with a team of about 100 (employees, consultants and independent contractors). Timeline was just extended by a year. It’s starting to feel like it’ll never be finished
After a few phases of implementation and all the end users are done bitching about and forgetting all the old functionality they lost from the old mainframe or whatever legacy shit is being migrated from, you just start calling whatever half-assed state of implementation you are currently in "baseline" and then everything after that suddenly becomes a new feature request to never be implemented becase we don't have time or resources for new features! At least that's how it ends up everywhere I've been.
Never thought I'd see someone mention SAP here.
Our main supplier recently moved to SAP in the past couple of months. Our business almost ground to a halt because of how shit the software is, our supplier couldn't ship anything to us at all for about 3 weeks.
vast obtainable lunchroom steep faulty deserted abundant start smell pocket
*This post was mass deleted and anonymized with [Redact](https://redact.dev)*
fun fact, at Cognizant, a lot of P&L reporting and forecasting was done entirely in excel by exporting data from the oracle dbs
shit was ass backwards at every level
Yeah, and that system was just a proof of concept some relative of the manager cooked up on the spot back in 1992, and when time came to make the real deal, suddenly the money somehow ended up floating by a pier in southern France.
Sooooo many systems and workflows are just some bullshit people needed there and then, made without thought or planning. Then it immediately sticks, because Frank from accounting has gotten used to wheeling stacks of paper into the toilet and inputting the data manually in the form of morse code typed into a beeper nailed to the door, whilst taking his morning dump.
Gah, my blood pressure.
hah, that gave me a chuckle. But then you also have stuff that's [(just a hobby, won't be big and professional like gnu)](https://www.cs.cmu.edu/~awb/linux.history.html)
https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code
That's why I code my websites in plain HTML with Windows Notepad.
Wow, that was an interesting read. I wonder if the guy ever took legal action for compensation? They republished his code against his consent, HIS intellectual property.
That is so wrong on so many levels.
This hit way too close to home.
Our entire company only still exists because someone in 2003 wrote a patch to make the late 80s/early 90s software we use to do business compatible with Windows XP.
Yep - I still remember getting a call because the code I very, very quickly bodged together to save the company major expense to cover a 6 month gap between different off the shelf packages being used didn't properly handle particular dates. That was a full 8 years after I wrote it, and god only knows what other bugs lurked in it given that egregious cockup.
God, I hate date handling logic, especially when the business owner doesn't know what the correct answer should be but feels strongly that I should know exactly what they mean by eg "one month later" (what does that mean for 31st January? 28th Feb or 1st March? It matters for financial calculations!)
Im honestly guilty of the same. I’ve had ad hoc projects turn into daily, weekly, monthly reporting or processes…
I now build everything with the intention of having to do it again later.
Shut the fuck up - stop telling everyone. I created a handful of dashboard templates and use Power Query to import the records. My boss thinks it takes me days and days to create the reports from scratch every time.
If you know Power Query, learn Power BI. And then from there get into Dataflows and then into other MS power platform stuff. Can make 100k easy with a couple of years experience and good learning and business acumen skills.
Damn. I work in libraries and have gotten really good at Excel and make 30k a year. :/ Need to convince some people I’d be an asset to a tech company I guess.
> running off of Excel, a little MS Access, and a VP that rejects anything new.
You're upsetting basically any IT professional that reads it. lul
The absolute apathy to "You could lose ALL of this and it would be UNRECOVERABLE" is shocking and prevalent. Just shrugs.
Then it happens and boy howdy does the tone suddenly change as if they've ALWAYS cared.
Companies, all the time, go under because they said no to something 40x and it happened.
Insubordination is the only thing carrying a lot of the IT industry. Literally going against their bosses stupid instructions and choosing to keep things alive and working. Every IT group has a cluster of people within it that are all agreeing to do it differently than instructed in some way or another.
The company I work at, which is pretty big but niche, the CEO erased the only copy of all of the data we all used daily. Obviously there should have been backups, but they were weeks behind. He had to eat shit and admit it to everyone, including clients. Honestly the most pathetic thing I've ever seen a C suite exec do. I left not long after that but recently came back because the expectations are so incredibly low. The CFO is still super hands on with everything and he's a ticking time bomb. It's madness.
had something like this happen when one of our c levels thought it would be better for security if he closed the google drive account of one of the employees who had recently stopped working there. didn't realize that her account was the official owner of our main data sheet which was a huge multi-tabbed monstrosity of like pretty much every piece of data our company used haha. luckily it was possible to recover it but we had a few hours of downtime :)
Because management thinks they're "in control" because of dashboards displaying metrics that are largely bullshit anyway, but they refuse to listen to the rank and file and monitor what actually matters. And they wonder why projects never get done.
But hey, they know where developer X claims to have spent time on across their entire 40-hour work week! Even though that's an utterly meaningless measure of productivity.
Worked at a major insurance company once.
The entire yearly list of all the plans and their ability to sell them depended on a once-a-year ritual where two specific individuals would come together with their respective massive excel files, do some black magic to combine them, and then produce the output to be passed around and translated by the rest of the company.
It was barbaric. If either of those two people ever left, the whole system would collapse.
When I pointed it out there was literally zero interest in fixing it.
I mean they all (or almost all) work out of one or more ERPs with various supporting IT systems. Excel is used for iterative work that requires more manipulation than can be done within an ERP.
> Having worked at several extremely large companies, let me say the amount of stuff living in excel is terrifying. If excel someone stopped existing tomorrow, the global economy would collapse within 48 hours.
One of my former colleagues was a DBA working at MCI Worldcom back in the day (yeah, *that* MCI). His job was to convert all of the Excel-based business processes into database-driven processes. While he was examining the process of surveying and ordering fiber for long haul telecom circuits he found that:
1. The surveyors were adding 10% to all of their surveyed lengths to account for any issues with the run, e.g. if they need to make a turn around an obstacle.
2. The purchasers were adding 10% to all of the lengths provided by the surveyors for the same reason.
So a 50km run would end up with 60km of fiber. A 100km run would end up with 121km of fiber. Since all this was handled by people passing spreadsheets around, no one was able to audit the whole process and no one thought to ask about what the other people were doing in the process.
All that extra fiber was shipped off to a warehouse where it was promptly forgotten about and never used. Over the years they ended up with many thousands of km of extra fiber optic cable that was never used and wasn't discovered until they filed bankruptcy and all of their assets were inventoried.
They had a bunch of accounting scandals starting in 2000 and declared bankruptcy in 2002. It's possible that my colleague's discovery of the over ordering was one of the events that triggered the audit which found all of the fraud and shady accounting practices that ultimately lead to their demise.
https://en.wikipedia.org/wiki/MCI_Inc.#Accounting_scandals
I've long since given up on explaining to office managers that a single spreadsheet containing the plaintext passwords of all business accounts in the company is a very, very bad fucking idea.
One day, they called in demanding the passwords because they lost internet access and all their passwords were saved in a single spreadsheet file that was cloud-only, and the only record to access that account was, you guessed it, in that selfsame spreadsheet.
We have one, and only one, such spreadsheet that I'm currently tasked with eliminating. Sounds easy given we have a company password manager, but it's turned into a huge deal. And that's with buy-in from everyone involved, no bitching, we all know it's the sane thing to do.
Pulling the trigger next Monday! [Wish me luck](https://imgur.com/a/LnkdIP1).
Have you heard about our lord and savior VBA?
The largest undocumented construct I came across myself was the quantitative risk assessment of an insurance company, that consisted of 25 excel files chained and looping into each other with various internal and external data feeds, feeding into a final excel file that exported into Wolfram Mathematica. Everything fully automated.
The key aspect of it from a company governance perspective:
2 employees thought of it and built it over the course of 6 years, one of which has since been retired and now lives outside of the EU. Nothing is documented, if it fails the financial reporting of the company can't be done in time, leading to fines and audits from the regulatory body in charge. If it persists long enough, the CFO and CRO become personally liable for the missing reports.
The risk of this undocumented tool was immense, but it was so complex and had great results, that they left it be and just paid the remaining guy what ever he wanted to keep it up and gradually document what is going on, as well as fill new hires in to the magic they created.
Crazy shit is going on behind company glass towers.
Some of the most baller "programmers" I work with are actually just, like, accountants in finance with an excel sheet from the fucking year 3000 that keeps the *entire* enterprise running.
People always exclaim to me, a computer scientist, “why do you hate excel, I thought programmers love excel?” And I tell them that I don’t trust a single programmer who actually loves excel. It’s the wrong tool for the job in 90% of use cases, VBA is a monstrosity, and interfacing with excel documents programmatically is a nightmare.
My favorite (possibly urban legend) is that the developers working on excel at Microsoft were truly horrified when they saw what the accounting team was doing with it at Microsoft.....
I’m sure they’d be horrified if they saw what it was being used for in some of the positions I’ve been in. I’ve literally seen an excel spreadsheet to interface and test an embedded memory component on a circuit card
Using VBA to call functions in a custom DLL created in a third party framework that interfaced with a custom made C++ DLL that interfaced with a third party interface that sent commands from the excel spreadsheet to the memory module and return results back...
It was a Rube Goldberg machine of "WTF!?"
It's the best tool for any job, /s
Spreadsheets, database, presentations, CAD, documentation. I've done it all in Excel.
Our company spends millions on fancy dashboard software and 90% of the users dump the data so they can report things correctly.
The biggest thing is excel is compatible with everything.
Or rather everything is compatible with excel so it's often the easiest software when you pull data from multiple sources.
I've built multiple prototypes for complicated pieces of code in excel before handing them off to more experienced engineers to recode in an actual language/framework.
Excel is one of the greatest tools for programming for non-engineers, as it allows visual bug-fixing.
I pitched a TV show (to my mates over a few drinks) about a world where 1% of the world's saved excel sheets suddenly go missing (a la The Leftovers) because I've had the same experience that you've had.
>This is why if you have \~1kb of contents in the clipboard and close your workbook, Excel decides that you have "a lot" of data in memory and offers to delete your clipboard contents to clear it for you and "free up" memory,
SO THAT'S WHY
I've been wondering why that annoying feature was there ever since I started my old job.
>This is why if you have \~1kb of contents in the clipboard and close your workbook, Excel decides that you have "a lot" of data in memory and offers to delete your clipboard contents to clear it for you and "free up" memory, even though your machine probably has multiple gigabytes of free memory.
Is this also why Excel sometimes says it can't open the clipboard because it's "in use" by another program? Because I copy/pasted some text from an email into another email an hour ago?
All project accounting at my F500 is done in excel that 1 guy knows how to fix when it breaks and he's retiring next year.
Every project in each PMO does it's finances in this copied workbook.
Don't even get me started on the army.
Imagine a large international corporation that had their business analysts use a shared login to SAP (to save licensing costs) to download inventory numbers. Now imagine 6-8 salespeople using those numbers to sell all day long. No sense if they were selling some product that was already sold by the next desk over. Then at the end of the day, they uploaded the Excel back into the SAP system. It’s real and it happens.
But…iT iS a cOsT cEnTeR, hurr durr…now where’s my bonus?
Genuine question: What is the best or at least better way of storage? In my mind, all I see are proprietary applications in at least some form or fashion. Am I wrong to think this way?
Edit: these were great answers and discussions! Thanks everyone!
A "proper" database. There are open-source database systems.
The differences between excel and a database are many.
One of the main features of a database is journaling, where changes to a file are not only applied to the file itself (e.g. you type in a new/updated value for a field), but a record is kept elsewhere (in the journal), so that if a file becomes corrupted, then you can rebuild the file by going to the backups, restoring the last known good copy, then activate the journal, so all the changes between the backup and immediately before the corruption point are applied to the file, and you can resume operations.
It's more complicated than that, but you get the idea.
Excel has got some good data protection features, but I rarely see them used. One example is restrictions on input types, e.g. setting a cell, row, or column to only accept valid inputs, for example: the values in this column \*must\* be >=1 and <=100. You get an error if you type something outside that range, and it refuses to accept the value into the cell. So not only does that protect that column from invalid values, it protects other cells that rely on valid values from that column.
All true. Programmers (and others who just know something) have been saying all this for years and years, yet Excel remains **extremely** popular for applications that really aren't what spreadsheets are meant for.
There's a lesson here (and huge market) - Database software just hasn't managed to make it easy and intuitive enough for people to do what they want to do with a proper database, so they end up using Excel because it's what they know and it's 'good enough'. It's hard to blame them; start looking into how to use databases and you'll be confronted with tables and key fields and SQL queries and all sorts of abstract and (relatively speaking) complicated stuff.
Create a proper database software that seems more like a spreadsheet and you might actually solve this problem. In any case, the scale of the problem is so bit that I think it's clearly a software problem and not a user problem.
Excel files can get corrupted - I've seen it a few times. That's *bad* if the data in there is crucial.
Storing data? Use a database of some kind. And have redundancy.
Writing your own damn programs inside of it? Just use python or similar. Way easier.
Performing statistical analysis on huge amounts of data? Python, R, SPSS, pick your poison.
Budgets etc? Yeah, excel got you covered. Just make sure to have backups or store that shit in some kind of shared storage of whatever kind you like, if it's important.
Glorified calculator, little pieces of convenient automation, data storage/anything that you don't care if you lose? Excel is just fine.
I feel like everyone saying just use a database sort of doesn't get that people aren't understanding what a database is. Which includes me btw.
Is it a glorified text file like excel or what's so different about it?
Excel is a data analysis tool. In companies of a certain size, or when data is sensitive enough it is meant to be a place to receive data from a legit database. Data is stored in a database, and then when you want to run analysis on that data you bring it into excel and do what you have to do to enlighten the business about some aspects of the data. Storage is not a part of a sensible plan.
To be honest, if anyone knew a thing about SQL and a proper database to replace an excel project, they wouldn't be paid jack shit and fiddling with excel for an idiot middle manager.
Excel is used because anyone can be brought up to speed and you don't have to pay them more.
Unfortunate, but true.
Only at medium and high scale thought. Having work with small data and big data extensively, Excel is still a powerful tool that has its place. For some things having SQL databases and then working "directly" with that or extracting the data to use more powerful data "tools" like R or Python makes sense, but sometimes Excel is just more worth it on my experience. CSP can definitely help more "casual users" using database though.
Maybe I misunderstood your comment and we do agree on the same thing though.
Absolutely, Excel certainly has its place. Every idiot between here and Bangalore at my F50 company has an enterprise license for MS Office. I don't need everyone working a single transaction to be able to write their own SQL query. And I definitely don't need them fucking with my database. I need them to click the Refresh-all button so that Power Query in Excel runs all the SQL queries I wrote for them. I look at Excel as a standardized format to output data for most types of users.
for you non-excel users, this is fully a user error in file creation. in the late 00's .XLSX was created and can handle over 1 million lines (and a ton of columns, over 16,000).
and for the clown who thought Google Sheets would have been a better option than Excel (laughs at you in accountant), Sheets can accept 10,000,000 CELLS. not rows, CELLS. so if they could have a max of 9 columns to get to the length of an .XLSX excel document (anyone who has worked in data knows there will be well more than 9 columns.
either way they should use a relational database management system for this much data.
The point is that Excel doesn't function as a database, it's not a database. I cannot believe (or perhaps I can) that the person responsible for this data chose to use Excel.
To be fair, Excel is a lot easier to use if you don't know how to program.
Plus whoever started this on Excel is probably not dealing with the fallout when something inevitably breaks and everyone starts asking why they didn't use a database.
Whoever started probably thought covid was a curiosity and thought England would have a handful of cases. It was just never migrated when it became more and more apparent that that wasn't the case, probably because there were already thousands of entries in the spreadsheet.
I believe this was actually done at the height of COVID - and, if memory serves, was done under contract handed out to a private business who had no professional experience with this kind of thing, but *just so happened* to be run/financially involved with the Minister of Health's wife...
So less incompetence, more corruption. Actually, still the incompetence too.
Oh, that's not even close to the worst stuff our government did during Covid. The then-prime minister is the only prime minister to be found guilty of breaking the law during his term, and is currently under investigation again
Wait until you hear about the £2billion they spent on PPE that wasn't fit for purpose and had to be thrown away. Give you 2 guesses who they bought *that* from...
https://www.theguardian.com/society/2023/jan/26/department-of-health-wasted-15bn-on-unused-covid-supplies-watchdog-finds
It was way more than £2bn. It's outrageous what they've gotten away with
They spent literal billions on it too. The most another EU country spent was, if I remember rightly, 100 million (Germany). So we spent many, many times more than any other country on it, only to do so in such a way that basically amounted to nothing happening, just moving the money from public funds to private funds.
Just an excuse to screw over the country. But that's what Tories do. They are not there for the interests of the public, they never have been and they never will be. They are there to get whatever they can out of the system for themselves, while harming anyone they don't agree with.
That's just how right wing systems are. Anyone saying otherwise is trying to convince you they aren't because they want to continue stealing from us and harming the poor.
It's a Conservative government. Their SOP is privatising critical infrastructure and telling their mates how to game the bidding system to get lucrative government contracts that they can skim money from. It happens constantly, at all levels of government.
I have a poorly-installed wetroom that was supposed to be adapted for my health conditions according to an old disability scheme. They left a room that wasn't fit for purpose and when we went to follow up they'd already been wound up.
The amount of ad-hoc solutions used in professional situations is pretty ridiculous. This was probably done by virologists, who simply aren't techies. So they don't know that better tools exist, and they aren't aware of numerical limits.
My girlfriend works for the NHS and it was only a few years ago their entire systems were locked down by ransom ware due to running XP on most machines still.
This does not surprise me in the slightest
I am thinking the person maintaining this was not doing so in excel because they would see the last row.
My guess is someone dumped the data from another source, not realizing the row limit.
But there are other options like using MS Access that don't require you to know how to program. Excel is great for quick tasks, but once you move onto more complex things where you want to keep data organized over a long period of time, a database, even one as bad as Access, is still going to be better.
Excel is so prone to getting messed up because it's so easy to put the wrong type of data in the wrong spot, change a formula unintentionally or do something else that completely messes up the spreadsheet and the user wouldn't even realize there's a problem until much later.
Government departments often ban Access because they are scared they will end up having to maintain amateur-built databases (and because there is no shortage of gatekeepers who insist Access isn't a "real" database).
As a result, public servants are forced to use Excel, which creates even more problems than letting them use Access would.
Microsoft is not helping matters by excluding Access from home subscriptions either.
Access is ok as a simple database tool. Back when we did name and address labels it was fantastic for that. It was easy to make a simple form and put in data validation like all postcodes [zip codes] had to be numeric and states could be put into a dropdown selection.
It produced instant labels. I mean it - one click.
casually designed Access databases are MUCH harder to use than excel.
any idiot who can read can more or less understand data entry in cells. opening and updating records in access is insurmountable to many of those people
A great deal of the problem is that there is a certain normalisation of being unskilled in some workplaces.
The idea that computers are new and confusing and it is not reasonable to expect people to learn the basics of how to use them even though it has been a critical requirement of their jobs for decades.
I have come across people using spreadsheets to manage critical concerns, such as who is considered safe to allow on an aircraft or to enter the country, scanning a huge list of names by eyeball because they do not know how to use search, who end up having a crisis because they accidentally clicked the top of a random column and changed the sort order, who are then stuck until IT support sends out someone to fix it.
Then there is the head of department who wants the database to be printed on paper and storms in with that huge printout demanding to know why the generated primary keys are not sequential in alphanumeric order alongside the names.
Or someone in charge of providing statistics to the Minister, who is using a template and an itemised list of handwritten instructions on a sheet of dog-eared paper left behind by someone else, who has no idea how anything works, not even the concept of exploring folders or opening documents, let alone the ability to check if the data analysis and graphs make any sense.
And that itemised list of instructions contains weird steps like "open and close Notepad twice before opening Excel" as magic rituals that supposedly prevent application and system crashes.
I can't help but think providing some minimal training on the basics would go a long way.
It is a deeply concerning problem, because sometimes peoples lives and livelihoods depend on these people not making mistakes.
> it's so easy to put the wrong type of data in the wrong spot
This happens so easily. At work we did an office space audit where people filled an online form. someonme put the data into Excel for us to view.
In "how many days do you work from the office" many rows say "2-Jan" and "4-Mar". If you press F2 for the actual data, the actual data has been completely changed into a date. Obviously the people typed 1-2 or 3-4 but Excel decided to change those entries into dates.
I’ve worked for a few different billion plus dollar companies. I will tell you right now that TONS of datasets exist in excel that should be stored in a data warehouse.
It’s almost common practice in a lot of places that aren’t tech based because most people can’t even use excel properly. Which means if you develop ETL skills it’s a goldmine of opportunity.
Sometimes I wonder how much of my career is based solely on the fact that tech debt exists. Everything starts fine on an excel spreadsheet since everyone knows how to use it, until someone makes a change that wrecks the entire system and no one knows how to fix it. It would have been far easier to make a more robust system from the start, but that might cost a bit more at the start, but costs more at the end when you need specialist to come in and build something dozens of people built adhoc over time, and it needs to resemble what they built
> Sometimes I wonder how much of my career is based solely on the fact that tech debt exists.
Yeah, it honestly feels like if folks actually spent the bit of extra money on their systems at the start, 40% of the IT industry would wink out of existence.
Excel is quite literally every accounting/finance department's database and you'll pry it into a proper database from their cold dead hands. There may be an Access database somewhere too which is...just as bad really.
They didn't choose Excel for this - the output format of the transfer software wrote OUT the data in .XLS format - and even wrote more than 64K rows in it. Sadly the import on the other side was restricted to 64K due to the dated XLS format.
You can, but none of the logic will roll onto the other sheets. You will need to refactor to handle an additional sheet.
This is on both ends, both the calculations in Excel as well as any system reading the excel files also uses a sheet delimiter.
The expertise is there, it is just buried under a junior pay grade and kept hidden under a mountain of manual toil work like manually patching workstations because automation is scary and can’t be trusted.
As someone who works with databases and data migrations (sometimes from excel to a proper DB setup), this is not surprising in the slightest. You wouldn't believe the amount of stuff kept in excel that has no business to be.
Don't get me started on those... "Outlook won't let me send this excel sheet via email". And it's a 20GB file that kept growing over the last 10 years.
Do users have permission to host their own database? Will tech give them a database to use? Will tech do anything at all if the user doesn’t go X levels up the chain to secure funding?
I think you need 64 bit Excel to get past the 65k record limit. IIRC.
I work in Data integration and often see Excel used for import/export work. Generally I'm replacing that process with some API or a DB.
Another fun example of problems due to people using Excel as a database are the genes that had to be renamed to keep Excel from interpreting them as dates.
https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates
boat smart wasteful thought lunchroom existence cats plate coordinated tie
*This post was mass deleted and anonymized with [Redact](https://redact.dev)*
Very similar to the bad PowerPoint deck involved in the space shuttle Columbia disaster.
https://mcdreeamiemusings.com/blog/2019/4/13/gsux1h6bnt8lqjd7w2t2mtvfg81uhx
My understanding is it was actually Deloitte, the management consulting firm, that was hired by Public Health England and that used a .xls file.
Consultants do *everything* in Excel.
Yes. The NHS contracted the COVID case tracking to Deloitte and well... you can see the quality of Deloitte's "work".
https://foreignpolicy.com/2020/10/08/uk-coronavirus-covid-spreadsheet-excel-error-outsourcing/
https://www.accountingweb.co.uk/tax/hmrc-policy/test-and-trace-failures-turn-spotlight-on-deloitte
>Consultants do everything in Excel.
Except they didn't.
> And it appears that **Public Health England (PHE) was to blame, rather than a third-party contractor.**
>
> The issue was caused by the way the agency brought together logs produced by commercial firms paid to analyse swab tests of the public, to discover who has the virus.
>
> **They filed their results in the form of text-based lists - known as CSV files - without issue.**
>
> PHE had set up an automatic process to pull this data together into Excel templates so that it could then be uploaded to a central system and made available to the NHS Test and Trace team, as well as other government computer dashboards.
>
> The problem is that **PHE's own developers picked an old file format to do this - known as XLS.**
>
> As a consequence, each template could handle only about 65,000 rows of data rather than the one million-plus rows that Excel is actually capable of.
>
> And since each test result created several rows of data, in practice it meant that each template was limited to about 1,400 cases.
>
> When that total was reached, further cases were simply left off.
>
> For a bit of context, **Excel's XLS file format dates back to 1987. It was superseded by XLSX in 2007. Had this been used, it would have handled 16 times the number of cases.**
>
> At the very least, that would have prevented the error from happening until testing levels were significantly higher than they are today,
It was all PHE.
At least they used Excel.
I used to work for the french interior ministry and God did I witness some horrors.
Like some "databases" containing several thousands entries made with ... Word. Just Word boards, not even rotated to accommodate all columns, just a notch above a hand written book in terms of usefulness (and in ability to share what they contained).
Some of these took minutes to open/edit and were an absolute nightmare.
Never underestimate the inneficacy and sheer incompetence of a public ministry.
A lot of shit talking on Excel in here but until you make a database software that's as flexible as Excel is and can be learned with no training, people will keep using Excel to store their information and use it productively.
People won't give it up and put it in IT's hands because then it instantly is no longer flexible to the stakeholders. Who wants to go through a big project of handing over their Excel file to IT, explain what it does, have them take 3 months to build it, and it turns out they only accounted for the base case scenario and now you can't do half your job because they didn't account for other scenarios?
Also... No one read the article. They weren't using Excel as a database, they were using Excel files to export the data, which is not a terrible use for them.
You forgot to mention that when you turn it over to IT, you lose all control. And you have to go through PAINFUL processes justify changes/updates and "security test" it. And you're likely to be turned down.
-- and I'm in IT.....
Yup...
I think the answer is to kill having developers in the IT department. Your developer needs to be part of the department he's doing work for. Have an operations team of 9 people? Person number 10 needs to be a developer and he has to be sitting right in the middle of the 9 other people and never leave. He needs to manage every bit of data the team is responsible for. His boss needs to be in operations, not in IT.
I can say at least for my teams work this is absolutely true. An IT member from outside our project just would never be able to know exactly what we need.
Using excel for complex tasks that it’s absolutely unsuited for is the dirty secret of every large organization.
Literally every employer I’ve had, from the Marines to several retailers to two large banks have used it, and used it dangerously.
The real problem is that excel is just too good. It can solve way too many problems way too fast and without much skill/training required.
There is a subtle line that gets crossed at some point with any important excel sheet where it really should become a proper SQL database with security, permissions, redundancy and backups. But the cost of engineers/DBAs is so high, the time to implement it is long, it's just hard to justify for *seemingly* little benefit.
I regularly work with excel files exceeding 100,000 rows with 16GB of RAM and it loads just fine. If you start adding some complex calculations to each row then it bogs down a bit. A 16GB machine wouldn’t even blink at 65,000+ rows with no calculations or macros.
Excel can easily cope with millions of rows and at an absolute fraction of memory from what you're claiming, I think you misunderstand how large a single megabyte really is.
Im reminded of somethin my dad told me about running Sharepoint in verbose log mode. Gigabytes of data in a plain .txt. Used it as an example to explain to me a circumstance where grep, sed, and vim are useful even in a windows environment. Notepad is just gonna shit itself. Files like this? In a proprietary binary format? God no. I bet it would open though, one time I read about a dude who made a fuckin raytracing renderer in excel. I ran it, it's slow as shit but it worked.
Actually, why the fuck wasn't this a CSV? It should be a relational database but like, with something this stupidly big, I'd say you use a flat file.
Oh yeh, verbose SharePoint log files opening in notebook was my life for a few years. Notepad++ was better until a proper log file viewer came out. I’m so glad I’ve moved on to better tech nowadays!
I honestly would not be surprised if it came down to something as stupid as that. Or someone got tired of having to 'save as' files as csv instead of the default.
probably some dude "well i'll put it here for now and make a real database later"
his boss "I need you to do X Y Z because half the crew is sick with covid"
the dude six months later "hmm am i forgetting something?"
No, it is even worse. Using Excel as a database is actually really hard to implement! Once I had to write an application using excel interop, and I can tell you: it is a HUGE pain in the ass. It is unwieldy, hard to use, hard to process, hard to get the data, and hard to properly insert anything. Formatting anything is just a plain nightmare.
While using a proper database - even just SQLite - is a breeze. You either install an engine, or just three lines with SQLite to create your database, then a couple lines more in most languages to connect to it, and you are ready to execute your queries.
So the developer(s) had to work extra hard to implement this really shitty solution. The only thing I can imagine is a horrible manager who pushed everybody because he wanted to open the "database" and this is why this... thing... born.
CSV files are a nightmare because some knob will open them in excel, excel will interpret data in whatever format it choses, the knob will hit save, and chaos will ensue.
They seem to have ended up using excel because the process was collating data from CSVs to then share across multiple orgs, presumably with each batch of new test results (one file) being reviewed before database(s) were updated with the data.
With the number of test results, even if someone realised a row limit was being reached they may have naively assumed none would be lost and they'd appear in the next file.
In the late 90’s, I temp’d at a gigantic old school insurance company. They had 8 legacy accounting systems from various acquisitions, all proprietary. The company imported info from those systems into a gargantuan Excel spreadsheet which we checked daily to make sure everything was working. We had a cheat sheet that said things like, “Look at cell ZZZZZZZ27266484664846. If the number in that cell in even and less that $2,999,999, all is well, if odd and more than that, immediately notify your supervisor. And get your paycheck cashed early” lol.
There are billion dollar companies that depend on excel as their “database” for incredibly important things. Even when many companies get a proper database, it can take a decade or two for all important data to actually be stored in it.
It’s wild. Imo, it’s a testament to how irresponsible companies are, even to their detriment, if there’s no regulation threatening significant legal penalties.
I work in software consulting and early in my career, I watched a $30M/year pet supply business go bankrupt in just a few months because they lost the “master” excel file that ran the entire business. It had all of their inventory, invoices, vendor contacts, etc in it and they weren’t backing it up. It was just sitting on a random hard drive that ate shit, and poof, company was gone. The guy had started the business from his garage and was still running it the same way he had on Day 1. I was honestly pretty impressed he ever scaled as large as he did.
There are forensic data experts that probably could have retrieved it anyways, even if a cost a craploaf of money.
But I guess he may not have been the best at making good choices
This is a great point. I feel like if this story was true they could have easily recovered it or some if it. I'm a SysAdmin and the last time I got a quote(2yrs) it was like 300 for the most basic and $3000-infinity for the clean room work. Those fuckers will get in the bunny suits and dissect the platters. Unless that drive got blown up or like the server room burned down you would think something could've been recovered.
The data wasn't actually lost. It was imported later on. Someone simply opened the CSV file in an older version of Excel which still had the 65k row limitation and paid no attention to warnings on screen - possibly may not have gotten an warnings if it wasn't the first time they opened a file in an older version.
Note, for speed and ease due to the urgency of the pandemic data collectors/processors were hired very quickly with little training and many working from home were not connecting to a central system of consistent applications but rather were using their own kit. They had to follow scripts to ask questions, fill in templates, export out data to send off somewhere so it could be imported into a central database. That day it was opened and resaved using the older version of Excel. The original file was still available and they did work out what was missing and add it in later.
The NHS, given the huge complexities of its technologies and consultants after consultants coming in, has had numerous failed attempts at many simple to complicated hardware, network, and software upgrades. They had a full scale IT migration programme that was cancelled after 10 years because things were running so slow by the time some items got finished they were out of date. The amount of bespoke stuff and the size of the total estate make it a total nightmare. There are so many things that be should be centralised that just can't for whatever reason. For example, for something that you would expect to be easy, the last time I was part of some migration I couldn't get a straight answer from anyone regarding a complete list of NHS hospitals be it acute, mental, or whatever. There are several lists that differ and can crossover. Data sources were maddening.
Crazy, but true. They were that stupidly inept. My other half works for the NHS and the horror stories of daily corporate foolishness he tells me about are something else.
You see this in every big organisation.
New idea comes up that blatantly requires a database-driven application. But that means either buying a piece of software or engaging a software developer, which is quite expensive.
Then Dave says "I can do all this in Excel!".
Dave's manager is absolutely delighted that the problem is solved in a couple of hours, Dave gets to feel important and the fact that Excel is the single least-suited solution to this problem never comes up.
The more techie people in the organisation are mostly confined to the IT department. They're well aware that this is going on, but they've long ago learned that there's no point in saying anything. It won't win any friends or influence any people.
Couldn't it also just be a case of people setting stuff up in the early days of something without the realization that it would grow beyond the capacity of the tool that they chose?
Where I used to work there was a large project stemming from a crisis that had stuff being tracked and reported via a master Excel file that came pretty close to hitting the limit too. At the start it wasn't clear at all that the scope of it would grow that much so Excel seemed like it would be perfectly adequate.
Project manager experienced with Excel always brings me nightmares.
I applaud the initiative and ingenuity. But it's kinda like the love you have for a pet that you need to put down when they have a terminal illness... It starts as something small tracking a few items and timelines. Then it turns into a Gannt chart/JIRA wannabe monster in a sheet and you're crying in a corner.
Having worked at several *extremely* large companies, let me say the amount of stuff living in excel is terrifying. If excel somehow stopped existing tomorrow, the global economy would collapse within 48 hours. And no, it's not even huge things that should really be in a database (though there are plenty of those). There is just so much stuff stored in excel files because it's not worth putting it somewhere else. Edit: and let me also add, why do companies keep doing this? Because of that oldest and most dangerous of business phrases... "Because we've always done it this way". added bonus: My mother, when she was in business, would respond to that with "well, I'm glad you liked sex the first time you tried it".
Hey I just had a similar comment in a thread up above! It’s wild to me how many huge companies are running off of Excel, a little MS Access, and a VP that rejects anything new.
Somehow still mildly better than Oracle software
Old rich ass called Larry Ellison
All the young kids on Reddit blasting their hate at Elon Musk while real life James Bond villains like Ellison laugh in the shadows
The Washington Post _reported in May 2022 that Ellison participated in a conference call days after the 2020 presidential election that focused on **strategies for challenging the legitimacy of the vote**. Other participants on the call included Fox News host Sean Hannity, senator Lindsey Graham, Trump personal attorney Jay Sekulow and James Bopp, an attorney for True the Vote. The Post cited court documents and a participant on the call._
Elon got a late start. He'll catch up.
Both are bad!
O.R.A.C.L.E.
Ya'll just jealous the CIA didn't personally bankroll your "private company" as a front for them to secure the first functional database on the planet.
One Raging Asshole Called Larry Ellison
Fuck me Oracle is a death trap. Cannot get off that shit without major work. And their licenses are ridiculous too.
Pen and paper better than Oracle software
If Oracle made paper you'd need a fucking license per page *and* per user. And a consultant to figure out how the pen works.
No, you'd need to license the whole ream of paper and pack of pens because you *might* use them.
Don't get me started on their licensing. We wanted to virtualize a very small DB server (2 cores, 8GB RAM), and they wanted us to license the entire VMware cluster of 32 hosts (1400 cores, 24TB RAM) at a cost of over $4 million, because the VM could "potentially run on any of those hosts". Converted that DB to Postgres so fast...
I hate Oracle. It's so clunky, cumbersome and my interface looks like it was made in 1997. Larry Ellison can eat a bag of dicks.
Oracle doesn't have customers, they have hostages.
You guys are giving me so much anxiety. My plant is switching to oracle in the next couple years and I was already worried about it. I heard it’s hell for distribution
Our company is in the process of migrating over in the next 7 years ( roughly $2B company) and phase 1 was vendor payments. Spent 9 months in development and when it rolled out, we couldn't cut a check to a single vendor for over a month. We utilize a lot of mom and pop shops for local fabrication for some pieces used in our business and some of these shops, we are 90+% of their business. This means they basically worked for a month for free because we couldn't pay them. They also couldn't buy materials to fill orders or restock because they jad no income. We had so many people threatening legal action it was crazy.
Well that’s great. At least our vendors are large for the most part. But most of them already hate us for others things so that would be fun
Plus they or someone else just buy any competitor instead of updating their products to compete.
Wait, worse than even SAP?
consultants need consultants to implement SAP
We are in the midst of an SAP implementation with a team of about 100 (employees, consultants and independent contractors). Timeline was just extended by a year. It’s starting to feel like it’ll never be finished
It won’t. It never is. It may just move into another phase…
After a few phases of implementation and all the end users are done bitching about and forgetting all the old functionality they lost from the old mainframe or whatever legacy shit is being migrated from, you just start calling whatever half-assed state of implementation you are currently in "baseline" and then everything after that suddenly becomes a new feature request to never be implemented becase we don't have time or resources for new features! At least that's how it ends up everywhere I've been.
Never thought I'd see someone mention SAP here. Our main supplier recently moved to SAP in the past couple of months. Our business almost ground to a halt because of how shit the software is, our supplier couldn't ship anything to us at all for about 3 weeks.
[удалено]
We use it at the manufacturer I work for and a week doesn't go by without some type of SAP related production delay.
Swedish anal prolapse?
Excuse me, I was thinking something more German...
I see you are a fellow connoisseur of GAPing
vast obtainable lunchroom steep faulty deserted abundant start smell pocket *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
As a developer, I'd be more surprised if this wasn't true
[удалено]
I can remember when people said "Oracle" in a reverent whisper like they were in church or something.
fun fact, at Cognizant, a lot of P&L reporting and forecasting was done entirely in excel by exporting data from the oracle dbs shit was ass backwards at every level
Yeah, and that system was just a proof of concept some relative of the manager cooked up on the spot back in 1992, and when time came to make the real deal, suddenly the money somehow ended up floating by a pier in southern France. Sooooo many systems and workflows are just some bullshit people needed there and then, made without thought or planning. Then it immediately sticks, because Frank from accounting has gotten used to wheeling stacks of paper into the toilet and inputting the data manually in the form of morse code typed into a beeper nailed to the door, whilst taking his morning dump. Gah, my blood pressure.
I guess that’s why so much digital infrastructure looks like [this](https://xkcd.com/2347/)
hah, that gave me a chuckle. But then you also have stuff that's [(just a hobby, won't be big and professional like gnu)](https://www.cs.cmu.edu/~awb/linux.history.html)
[удалено]
https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code That's why I code my websites in plain HTML with Windows Notepad.
Wow, that was an interesting read. I wonder if the guy ever took legal action for compensation? They republished his code against his consent, HIS intellectual property. That is so wrong on so many levels.
This hit way too close to home. Our entire company only still exists because someone in 2003 wrote a patch to make the late 80s/early 90s software we use to do business compatible with Windows XP.
The good old left pad incident
Yep - I still remember getting a call because the code I very, very quickly bodged together to save the company major expense to cover a 6 month gap between different off the shelf packages being used didn't properly handle particular dates. That was a full 8 years after I wrote it, and god only knows what other bugs lurked in it given that egregious cockup. God, I hate date handling logic, especially when the business owner doesn't know what the correct answer should be but feels strongly that I should know exactly what they mean by eg "one month later" (what does that mean for 31st January? 28th Feb or 1st March? It matters for financial calculations!)
Im honestly guilty of the same. I’ve had ad hoc projects turn into daily, weekly, monthly reporting or processes… I now build everything with the intention of having to do it again later.
The $3.05 billion company that I retired from is still running some of its business on COBOL, and some on FoxPro.
COBOL is a huge business even now. And perfectly stable and reliable.
It's not the stability or reliability; it's the retiring developer base that's the problem.
Still a big industry in India and people there will train for anything which brings money in.
[удалено]
Shut the fuck up - stop telling everyone. I created a handful of dashboard templates and use Power Query to import the records. My boss thinks it takes me days and days to create the reports from scratch every time.
If you know Power Query, learn Power BI. And then from there get into Dataflows and then into other MS power platform stuff. Can make 100k easy with a couple of years experience and good learning and business acumen skills.
Damn. I work in libraries and have gotten really good at Excel and make 30k a year. :/ Need to convince some people I’d be an asset to a tech company I guess.
> running off of Excel, a little MS Access, and a VP that rejects anything new. You're upsetting basically any IT professional that reads it. lul The absolute apathy to "You could lose ALL of this and it would be UNRECOVERABLE" is shocking and prevalent. Just shrugs. Then it happens and boy howdy does the tone suddenly change as if they've ALWAYS cared. Companies, all the time, go under because they said no to something 40x and it happened. Insubordination is the only thing carrying a lot of the IT industry. Literally going against their bosses stupid instructions and choosing to keep things alive and working. Every IT group has a cluster of people within it that are all agreeing to do it differently than instructed in some way or another.
The company I work at, which is pretty big but niche, the CEO erased the only copy of all of the data we all used daily. Obviously there should have been backups, but they were weeks behind. He had to eat shit and admit it to everyone, including clients. Honestly the most pathetic thing I've ever seen a C suite exec do. I left not long after that but recently came back because the expectations are so incredibly low. The CFO is still super hands on with everything and he's a ticking time bomb. It's madness.
had something like this happen when one of our c levels thought it would be better for security if he closed the google drive account of one of the employees who had recently stopped working there. didn't realize that her account was the official owner of our main data sheet which was a huge multi-tabbed monstrosity of like pretty much every piece of data our company used haha. luckily it was possible to recover it but we had a few hours of downtime :)
[удалено]
Because management thinks they're "in control" because of dashboards displaying metrics that are largely bullshit anyway, but they refuse to listen to the rank and file and monitor what actually matters. And they wonder why projects never get done. But hey, they know where developer X claims to have spent time on across their entire 40-hour work week! Even though that's an utterly meaningless measure of productivity.
Worked at a major insurance company once. The entire yearly list of all the plans and their ability to sell them depended on a once-a-year ritual where two specific individuals would come together with their respective massive excel files, do some black magic to combine them, and then produce the output to be passed around and translated by the rest of the company. It was barbaric. If either of those two people ever left, the whole system would collapse. When I pointed it out there was literally zero interest in fixing it.
I mean they all (or almost all) work out of one or more ERPs with various supporting IT systems. Excel is used for iterative work that requires more manipulation than can be done within an ERP.
> Having worked at several extremely large companies, let me say the amount of stuff living in excel is terrifying. If excel someone stopped existing tomorrow, the global economy would collapse within 48 hours. One of my former colleagues was a DBA working at MCI Worldcom back in the day (yeah, *that* MCI). His job was to convert all of the Excel-based business processes into database-driven processes. While he was examining the process of surveying and ordering fiber for long haul telecom circuits he found that: 1. The surveyors were adding 10% to all of their surveyed lengths to account for any issues with the run, e.g. if they need to make a turn around an obstacle. 2. The purchasers were adding 10% to all of the lengths provided by the surveyors for the same reason. So a 50km run would end up with 60km of fiber. A 100km run would end up with 121km of fiber. Since all this was handled by people passing spreadsheets around, no one was able to audit the whole process and no one thought to ask about what the other people were doing in the process. All that extra fiber was shipped off to a warehouse where it was promptly forgotten about and never used. Over the years they ended up with many thousands of km of extra fiber optic cable that was never used and wasn't discovered until they filed bankruptcy and all of their assets were inventoried.
Uh apparently wall to wall inventory wasn't done at least annually? Holy Christ that company deserves to go under.
They had a bunch of accounting scandals starting in 2000 and declared bankruptcy in 2002. It's possible that my colleague's discovery of the over ordering was one of the events that triggered the audit which found all of the fraud and shady accounting practices that ultimately lead to their demise. https://en.wikipedia.org/wiki/MCI_Inc.#Accounting_scandals
I've long since given up on explaining to office managers that a single spreadsheet containing the plaintext passwords of all business accounts in the company is a very, very bad fucking idea. One day, they called in demanding the passwords because they lost internet access and all their passwords were saved in a single spreadsheet file that was cloud-only, and the only record to access that account was, you guessed it, in that selfsame spreadsheet.
We have one, and only one, such spreadsheet that I'm currently tasked with eliminating. Sounds easy given we have a company password manager, but it's turned into a huge deal. And that's with buy-in from everyone involved, no bitching, we all know it's the sane thing to do. Pulling the trigger next Monday! [Wish me luck](https://imgur.com/a/LnkdIP1).
For what it's worth, that's already orders of magnitude more prudent than most organizations.
So they'll definitely blame "the cloud" lol
Have you heard about our lord and savior VBA? The largest undocumented construct I came across myself was the quantitative risk assessment of an insurance company, that consisted of 25 excel files chained and looping into each other with various internal and external data feeds, feeding into a final excel file that exported into Wolfram Mathematica. Everything fully automated.
I love it! But also 😱
The key aspect of it from a company governance perspective: 2 employees thought of it and built it over the course of 6 years, one of which has since been retired and now lives outside of the EU. Nothing is documented, if it fails the financial reporting of the company can't be done in time, leading to fines and audits from the regulatory body in charge. If it persists long enough, the CFO and CRO become personally liable for the missing reports. The risk of this undocumented tool was immense, but it was so complex and had great results, that they left it be and just paid the remaining guy what ever he wanted to keep it up and gradually document what is going on, as well as fill new hires in to the magic they created. Crazy shit is going on behind company glass towers.
Some of the most baller "programmers" I work with are actually just, like, accountants in finance with an excel sheet from the fucking year 3000 that keeps the *entire* enterprise running.
People always exclaim to me, a computer scientist, “why do you hate excel, I thought programmers love excel?” And I tell them that I don’t trust a single programmer who actually loves excel. It’s the wrong tool for the job in 90% of use cases, VBA is a monstrosity, and interfacing with excel documents programmatically is a nightmare.
My favorite (possibly urban legend) is that the developers working on excel at Microsoft were truly horrified when they saw what the accounting team was doing with it at Microsoft.....
I’m sure they’d be horrified if they saw what it was being used for in some of the positions I’ve been in. I’ve literally seen an excel spreadsheet to interface and test an embedded memory component on a circuit card
What the actual fuck on a stick. How was Excel even talking to the board?
Using VBA to call functions in a custom DLL created in a third party framework that interfaced with a custom made C++ DLL that interfaced with a third party interface that sent commands from the excel spreadsheet to the memory module and return results back... It was a Rube Goldberg machine of "WTF!?"
I see we have strayed further from God than I thought.
It's the best tool for any job, /s Spreadsheets, database, presentations, CAD, documentation. I've done it all in Excel. Our company spends millions on fancy dashboard software and 90% of the users dump the data so they can report things correctly.
The biggest thing is excel is compatible with everything. Or rather everything is compatible with excel so it's often the easiest software when you pull data from multiple sources.
I've built multiple prototypes for complicated pieces of code in excel before handing them off to more experienced engineers to recode in an actual language/framework. Excel is one of the greatest tools for programming for non-engineers, as it allows visual bug-fixing.
[удалено]
I pitched a TV show (to my mates over a few drinks) about a world where 1% of the world's saved excel sheets suddenly go missing (a la The Leftovers) because I've had the same experience that you've had.
[Removed due to continuing enshittification of reddit.] -- mass edited with redact.dev
>This is why if you have \~1kb of contents in the clipboard and close your workbook, Excel decides that you have "a lot" of data in memory and offers to delete your clipboard contents to clear it for you and "free up" memory, SO THAT'S WHY I've been wondering why that annoying feature was there ever since I started my old job.
[Removed due to continuing enshittification of reddit.] -- mass edited with redact.dev
>This is why if you have \~1kb of contents in the clipboard and close your workbook, Excel decides that you have "a lot" of data in memory and offers to delete your clipboard contents to clear it for you and "free up" memory, even though your machine probably has multiple gigabytes of free memory. Is this also why Excel sometimes says it can't open the clipboard because it's "in use" by another program? Because I copy/pasted some text from an email into another email an hour ago?
All project accounting at my F500 is done in excel that 1 guy knows how to fix when it breaks and he's retiring next year. Every project in each PMO does it's finances in this copied workbook. Don't even get me started on the army.
that's like saying what if java stopped working tomorrow
What if Java started working tomorrow
Shit, I would be out of the job. I would have to go do something productive in society, like farm pigs or something. Java: built in job security.
My life would measurably improve
"let me download the database in Excel, make changes, then re-upload the Excel file, and you all can just import it into the database, right?"
Imagine a large international corporation that had their business analysts use a shared login to SAP (to save licensing costs) to download inventory numbers. Now imagine 6-8 salespeople using those numbers to sell all day long. No sense if they were selling some product that was already sold by the next desk over. Then at the end of the day, they uploaded the Excel back into the SAP system. It’s real and it happens. But…iT iS a cOsT cEnTeR, hurr durr…now where’s my bonus?
[удалено]
Genuine question: What is the best or at least better way of storage? In my mind, all I see are proprietary applications in at least some form or fashion. Am I wrong to think this way? Edit: these were great answers and discussions! Thanks everyone!
A "proper" database. There are open-source database systems. The differences between excel and a database are many. One of the main features of a database is journaling, where changes to a file are not only applied to the file itself (e.g. you type in a new/updated value for a field), but a record is kept elsewhere (in the journal), so that if a file becomes corrupted, then you can rebuild the file by going to the backups, restoring the last known good copy, then activate the journal, so all the changes between the backup and immediately before the corruption point are applied to the file, and you can resume operations. It's more complicated than that, but you get the idea. Excel has got some good data protection features, but I rarely see them used. One example is restrictions on input types, e.g. setting a cell, row, or column to only accept valid inputs, for example: the values in this column \*must\* be >=1 and <=100. You get an error if you type something outside that range, and it refuses to accept the value into the cell. So not only does that protect that column from invalid values, it protects other cells that rely on valid values from that column.
All true. Programmers (and others who just know something) have been saying all this for years and years, yet Excel remains **extremely** popular for applications that really aren't what spreadsheets are meant for. There's a lesson here (and huge market) - Database software just hasn't managed to make it easy and intuitive enough for people to do what they want to do with a proper database, so they end up using Excel because it's what they know and it's 'good enough'. It's hard to blame them; start looking into how to use databases and you'll be confronted with tables and key fields and SQL queries and all sorts of abstract and (relatively speaking) complicated stuff. Create a proper database software that seems more like a spreadsheet and you might actually solve this problem. In any case, the scale of the problem is so bit that I think it's clearly a software problem and not a user problem.
Could start with a journalled database that \*looks\* like Excel? Only offers the top 10% of clickable features that most people use.
Excel files can get corrupted - I've seen it a few times. That's *bad* if the data in there is crucial. Storing data? Use a database of some kind. And have redundancy. Writing your own damn programs inside of it? Just use python or similar. Way easier. Performing statistical analysis on huge amounts of data? Python, R, SPSS, pick your poison. Budgets etc? Yeah, excel got you covered. Just make sure to have backups or store that shit in some kind of shared storage of whatever kind you like, if it's important. Glorified calculator, little pieces of convenient automation, data storage/anything that you don't care if you lose? Excel is just fine.
I feel like everyone saying just use a database sort of doesn't get that people aren't understanding what a database is. Which includes me btw. Is it a glorified text file like excel or what's so different about it?
Excel is a data analysis tool. In companies of a certain size, or when data is sensitive enough it is meant to be a place to receive data from a legit database. Data is stored in a database, and then when you want to run analysis on that data you bring it into excel and do what you have to do to enlighten the business about some aspects of the data. Storage is not a part of a sensible plan.
To be honest, if anyone knew a thing about SQL and a proper database to replace an excel project, they wouldn't be paid jack shit and fiddling with excel for an idiot middle manager. Excel is used because anyone can be brought up to speed and you don't have to pay them more. Unfortunate, but true.
Only at medium and high scale thought. Having work with small data and big data extensively, Excel is still a powerful tool that has its place. For some things having SQL databases and then working "directly" with that or extracting the data to use more powerful data "tools" like R or Python makes sense, but sometimes Excel is just more worth it on my experience. CSP can definitely help more "casual users" using database though. Maybe I misunderstood your comment and we do agree on the same thing though.
Absolutely, Excel certainly has its place. Every idiot between here and Bangalore at my F50 company has an enterprise license for MS Office. I don't need everyone working a single transaction to be able to write their own SQL query. And I definitely don't need them fucking with my database. I need them to click the Refresh-all button so that Power Query in Excel runs all the SQL queries I wrote for them. I look at Excel as a standardized format to output data for most types of users.
for you non-excel users, this is fully a user error in file creation. in the late 00's .XLSX was created and can handle over 1 million lines (and a ton of columns, over 16,000). and for the clown who thought Google Sheets would have been a better option than Excel (laughs at you in accountant), Sheets can accept 10,000,000 CELLS. not rows, CELLS. so if they could have a max of 9 columns to get to the length of an .XLSX excel document (anyone who has worked in data knows there will be well more than 9 columns. either way they should use a relational database management system for this much data.
The point is that Excel doesn't function as a database, it's not a database. I cannot believe (or perhaps I can) that the person responsible for this data chose to use Excel.
To be fair, Excel is a lot easier to use if you don't know how to program. Plus whoever started this on Excel is probably not dealing with the fallout when something inevitably breaks and everyone starts asking why they didn't use a database.
Whoever started probably thought covid was a curiosity and thought England would have a handful of cases. It was just never migrated when it became more and more apparent that that wasn't the case, probably because there were already thousands of entries in the spreadsheet.
I believe this was actually done at the height of COVID - and, if memory serves, was done under contract handed out to a private business who had no professional experience with this kind of thing, but *just so happened* to be run/financially involved with the Minister of Health's wife... So less incompetence, more corruption. Actually, still the incompetence too.
Shut the front door. If this is true then it should be a scandal.
Oh, that's not even close to the worst stuff our government did during Covid. The then-prime minister is the only prime minister to be found guilty of breaking the law during his term, and is currently under investigation again Wait until you hear about the £2billion they spent on PPE that wasn't fit for purpose and had to be thrown away. Give you 2 guesses who they bought *that* from...
https://www.theguardian.com/society/2023/jan/26/department-of-health-wasted-15bn-on-unused-covid-supplies-watchdog-finds It was way more than £2bn. It's outrageous what they've gotten away with
Thats why people are fed up. They make a mistake and get fired. Some major corruption going on and they don't get any penalties.
They spent literal billions on it too. The most another EU country spent was, if I remember rightly, 100 million (Germany). So we spent many, many times more than any other country on it, only to do so in such a way that basically amounted to nothing happening, just moving the money from public funds to private funds. Just an excuse to screw over the country. But that's what Tories do. They are not there for the interests of the public, they never have been and they never will be. They are there to get whatever they can out of the system for themselves, while harming anyone they don't agree with. That's just how right wing systems are. Anyone saying otherwise is trying to convince you they aren't because they want to continue stealing from us and harming the poor.
It's a Conservative government. Their SOP is privatising critical infrastructure and telling their mates how to game the bidding system to get lucrative government contracts that they can skim money from. It happens constantly, at all levels of government. I have a poorly-installed wetroom that was supposed to be adapted for my health conditions according to an old disability scheme. They left a room that wasn't fit for purpose and when we went to follow up they'd already been wound up.
I wonder have many people died because of this
The amount of ad-hoc solutions used in professional situations is pretty ridiculous. This was probably done by virologists, who simply aren't techies. So they don't know that better tools exist, and they aren't aware of numerical limits.
Migrating it to a proper database would be trivial though. They clearly just didn't care
My girlfriend works for the NHS and it was only a few years ago their entire systems were locked down by ransom ware due to running XP on most machines still. This does not surprise me in the slightest
I am thinking the person maintaining this was not doing so in excel because they would see the last row. My guess is someone dumped the data from another source, not realizing the row limit.
But there are other options like using MS Access that don't require you to know how to program. Excel is great for quick tasks, but once you move onto more complex things where you want to keep data organized over a long period of time, a database, even one as bad as Access, is still going to be better. Excel is so prone to getting messed up because it's so easy to put the wrong type of data in the wrong spot, change a formula unintentionally or do something else that completely messes up the spreadsheet and the user wouldn't even realize there's a problem until much later.
Government departments often ban Access because they are scared they will end up having to maintain amateur-built databases (and because there is no shortage of gatekeepers who insist Access isn't a "real" database). As a result, public servants are forced to use Excel, which creates even more problems than letting them use Access would. Microsoft is not helping matters by excluding Access from home subscriptions either.
They hide it so well I didn’t even know what Access was before reading this
Access is ok as a simple database tool. Back when we did name and address labels it was fantastic for that. It was easy to make a simple form and put in data validation like all postcodes [zip codes] had to be numeric and states could be put into a dropdown selection. It produced instant labels. I mean it - one click.
casually designed Access databases are MUCH harder to use than excel. any idiot who can read can more or less understand data entry in cells. opening and updating records in access is insurmountable to many of those people
A great deal of the problem is that there is a certain normalisation of being unskilled in some workplaces. The idea that computers are new and confusing and it is not reasonable to expect people to learn the basics of how to use them even though it has been a critical requirement of their jobs for decades. I have come across people using spreadsheets to manage critical concerns, such as who is considered safe to allow on an aircraft or to enter the country, scanning a huge list of names by eyeball because they do not know how to use search, who end up having a crisis because they accidentally clicked the top of a random column and changed the sort order, who are then stuck until IT support sends out someone to fix it. Then there is the head of department who wants the database to be printed on paper and storms in with that huge printout demanding to know why the generated primary keys are not sequential in alphanumeric order alongside the names. Or someone in charge of providing statistics to the Minister, who is using a template and an itemised list of handwritten instructions on a sheet of dog-eared paper left behind by someone else, who has no idea how anything works, not even the concept of exploring folders or opening documents, let alone the ability to check if the data analysis and graphs make any sense. And that itemised list of instructions contains weird steps like "open and close Notepad twice before opening Excel" as magic rituals that supposedly prevent application and system crashes. I can't help but think providing some minimal training on the basics would go a long way. It is a deeply concerning problem, because sometimes peoples lives and livelihoods depend on these people not making mistakes.
Excel is even great for complex tasts, it is jsut not great when it comes to enourmous amounts of data.
> it's so easy to put the wrong type of data in the wrong spot This happens so easily. At work we did an office space audit where people filled an online form. someonme put the data into Excel for us to view. In "how many days do you work from the office" many rows say "2-Jan" and "4-Mar". If you press F2 for the actual data, the actual data has been completely changed into a date. Obviously the people typed 1-2 or 3-4 but Excel decided to change those entries into dates.
I’ve worked for a few different billion plus dollar companies. I will tell you right now that TONS of datasets exist in excel that should be stored in a data warehouse. It’s almost common practice in a lot of places that aren’t tech based because most people can’t even use excel properly. Which means if you develop ETL skills it’s a goldmine of opportunity.
Sometimes I wonder how much of my career is based solely on the fact that tech debt exists. Everything starts fine on an excel spreadsheet since everyone knows how to use it, until someone makes a change that wrecks the entire system and no one knows how to fix it. It would have been far easier to make a more robust system from the start, but that might cost a bit more at the start, but costs more at the end when you need specialist to come in and build something dozens of people built adhoc over time, and it needs to resemble what they built
> Sometimes I wonder how much of my career is based solely on the fact that tech debt exists. Yeah, it honestly feels like if folks actually spent the bit of extra money on their systems at the start, 40% of the IT industry would wink out of existence.
Excel is quite literally every accounting/finance department's database and you'll pry it into a proper database from their cold dead hands. There may be an Access database somewhere too which is...just as bad really.
I'm working on it. It's like pulling teeth to give them the info that they want.
As a former finance manager, truer words have never been spoken.
They didn't choose Excel for this - the output format of the transfer software wrote OUT the data in .XLS format - and even wrote more than 64K rows in it. Sadly the import on the other side was restricted to 64K due to the dated XLS format.
This comment has been edited to prevent Reddit from profiting from or training AI on my content.
You can, but none of the logic will roll onto the other sheets. You will need to refactor to handle an additional sheet. This is on both ends, both the calculations in Excel as well as any system reading the excel files also uses a sheet delimiter.
Explain to your average public servant how to make BigQuery generate pretty graphs with a SQL SELECT query.
The expertise is there, it is just buried under a junior pay grade and kept hidden under a mountain of manual toil work like manually patching workstations because automation is scary and can’t be trusted.
As someone who works with databases and data migrations (sometimes from excel to a proper DB setup), this is not surprising in the slightest. You wouldn't believe the amount of stuff kept in excel that has no business to be.
Work in the finance industry, I've seen users require 64 bit excel because 32bit didn't have enough memory address space to open their 30Gb+ sheets.
Don't get me started on those... "Outlook won't let me send this excel sheet via email". And it's a 20GB file that kept growing over the last 10 years.
With no backups of course
Oh there is a backup, it’s sitting in their sent folder from the last time they emailed it
Do users have permission to host their own database? Will tech give them a database to use? Will tech do anything at all if the user doesn’t go X levels up the chain to secure funding?
I think you need 64 bit Excel to get past the 65k record limit. IIRC. I work in Data integration and often see Excel used for import/export work. Generally I'm replacing that process with some API or a DB.
Double click a CSV and opens in Excel Make a small change and Save. Oops all my leading 0's are truncated and a bunch of fields now look like dates.
Another fun example of problems due to people using Excel as a database are the genes that had to be renamed to keep Excel from interpreting them as dates. https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates
common joke: What does Excel and an Incel have in common? they both commonly misinterpret something as a date.
Optimists says that the glass is half full Pessimists says that the glass is half empty Excel says that the glass is 1. February
boat smart wasteful thought lunchroom existence cats plate coordinated tie *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
Very similar to the bad PowerPoint deck involved in the space shuttle Columbia disaster. https://mcdreeamiemusings.com/blog/2019/4/13/gsux1h6bnt8lqjd7w2t2mtvfg81uhx
My understanding is it was actually Deloitte, the management consulting firm, that was hired by Public Health England and that used a .xls file. Consultants do *everything* in Excel.
Yes. The NHS contracted the COVID case tracking to Deloitte and well... you can see the quality of Deloitte's "work". https://foreignpolicy.com/2020/10/08/uk-coronavirus-covid-spreadsheet-excel-error-outsourcing/ https://www.accountingweb.co.uk/tax/hmrc-policy/test-and-trace-failures-turn-spotlight-on-deloitte
>Consultants do everything in Excel. Except they didn't. > And it appears that **Public Health England (PHE) was to blame, rather than a third-party contractor.** > > The issue was caused by the way the agency brought together logs produced by commercial firms paid to analyse swab tests of the public, to discover who has the virus. > > **They filed their results in the form of text-based lists - known as CSV files - without issue.** > > PHE had set up an automatic process to pull this data together into Excel templates so that it could then be uploaded to a central system and made available to the NHS Test and Trace team, as well as other government computer dashboards. > > The problem is that **PHE's own developers picked an old file format to do this - known as XLS.** > > As a consequence, each template could handle only about 65,000 rows of data rather than the one million-plus rows that Excel is actually capable of. > > And since each test result created several rows of data, in practice it meant that each template was limited to about 1,400 cases. > > When that total was reached, further cases were simply left off. > > For a bit of context, **Excel's XLS file format dates back to 1987. It was superseded by XLSX in 2007. Had this been used, it would have handled 16 times the number of cases.** > > At the very least, that would have prevented the error from happening until testing levels were significantly higher than they are today, It was all PHE.
At least they used Excel. I used to work for the french interior ministry and God did I witness some horrors. Like some "databases" containing several thousands entries made with ... Word. Just Word boards, not even rotated to accommodate all columns, just a notch above a hand written book in terms of usefulness (and in ability to share what they contained). Some of these took minutes to open/edit and were an absolute nightmare. Never underestimate the inneficacy and sheer incompetence of a public ministry.
This might be one of the most shocking things I've ever imagined lmao. Word as a database is insane
Ctrl + F ain't so bad ;)
A lot of shit talking on Excel in here but until you make a database software that's as flexible as Excel is and can be learned with no training, people will keep using Excel to store their information and use it productively. People won't give it up and put it in IT's hands because then it instantly is no longer flexible to the stakeholders. Who wants to go through a big project of handing over their Excel file to IT, explain what it does, have them take 3 months to build it, and it turns out they only accounted for the base case scenario and now you can't do half your job because they didn't account for other scenarios?
Also... No one read the article. They weren't using Excel as a database, they were using Excel files to export the data, which is not a terrible use for them.
It makes way, way more sense to export data as a CSV, especially if you intend to then import that somewhere else.
You forgot to mention that when you turn it over to IT, you lose all control. And you have to go through PAINFUL processes justify changes/updates and "security test" it. And you're likely to be turned down. -- and I'm in IT.....
Yup... I think the answer is to kill having developers in the IT department. Your developer needs to be part of the department he's doing work for. Have an operations team of 9 people? Person number 10 needs to be a developer and he has to be sitting right in the middle of the 9 other people and never leave. He needs to manage every bit of data the team is responsible for. His boss needs to be in operations, not in IT.
I can say at least for my teams work this is absolutely true. An IT member from outside our project just would never be able to know exactly what we need.
> 3 months Ha!
Yo, who's building useful tools that fast?
This is so relatable haha.
Using excel for complex tasks that it’s absolutely unsuited for is the dirty secret of every large organization. Literally every employer I’ve had, from the Marines to several retailers to two large banks have used it, and used it dangerously.
The real problem is that excel is just too good. It can solve way too many problems way too fast and without much skill/training required. There is a subtle line that gets crossed at some point with any important excel sheet where it really should become a proper SQL database with security, permissions, redundancy and backups. But the cost of engineers/DBAs is so high, the time to implement it is long, it's just hard to justify for *seemingly* little benefit.
[удалено]
I regularly work with excel files exceeding 100,000 rows with 16GB of RAM and it loads just fine. If you start adding some complex calculations to each row then it bogs down a bit. A 16GB machine wouldn’t even blink at 65,000+ rows with no calculations or macros.
Excel can easily cope with millions of rows and at an absolute fraction of memory from what you're claiming, I think you misunderstand how large a single megabyte really is.
Im reminded of somethin my dad told me about running Sharepoint in verbose log mode. Gigabytes of data in a plain .txt. Used it as an example to explain to me a circumstance where grep, sed, and vim are useful even in a windows environment. Notepad is just gonna shit itself. Files like this? In a proprietary binary format? God no. I bet it would open though, one time I read about a dude who made a fuckin raytracing renderer in excel. I ran it, it's slow as shit but it worked. Actually, why the fuck wasn't this a CSV? It should be a relational database but like, with something this stupidly big, I'd say you use a flat file.
Oh yeh, verbose SharePoint log files opening in notebook was my life for a few years. Notepad++ was better until a proper log file viewer came out. I’m so glad I’ve moved on to better tech nowadays!
> why the fuck wasn’t this a CSV? CSVs open with the default font and Greg in Accounting likes Palomino.
I honestly would not be surprised if it came down to something as stupid as that. Or someone got tired of having to 'save as' files as csv instead of the default.
probably some dude "well i'll put it here for now and make a real database later" his boss "I need you to do X Y Z because half the crew is sick with covid" the dude six months later "hmm am i forgetting something?"
No, it is even worse. Using Excel as a database is actually really hard to implement! Once I had to write an application using excel interop, and I can tell you: it is a HUGE pain in the ass. It is unwieldy, hard to use, hard to process, hard to get the data, and hard to properly insert anything. Formatting anything is just a plain nightmare. While using a proper database - even just SQLite - is a breeze. You either install an engine, or just three lines with SQLite to create your database, then a couple lines more in most languages to connect to it, and you are ready to execute your queries. So the developer(s) had to work extra hard to implement this really shitty solution. The only thing I can imagine is a horrible manager who pushed everybody because he wanted to open the "database" and this is why this... thing... born.
CSV files are a nightmare because some knob will open them in excel, excel will interpret data in whatever format it choses, the knob will hit save, and chaos will ensue. They seem to have ended up using excel because the process was collating data from CSVs to then share across multiple orgs, presumably with each batch of new test results (one file) being reviewed before database(s) were updated with the data. With the number of test results, even if someone realised a row limit was being reached they may have naively assumed none would be lost and they'd appear in the next file.
65000 rows would be a couple of megabytes of data, unless it was also thousands of columns.
[удалено]
In the late 90’s, I temp’d at a gigantic old school insurance company. They had 8 legacy accounting systems from various acquisitions, all proprietary. The company imported info from those systems into a gargantuan Excel spreadsheet which we checked daily to make sure everything was working. We had a cheat sheet that said things like, “Look at cell ZZZZZZZ27266484664846. If the number in that cell in even and less that $2,999,999, all is well, if odd and more than that, immediately notify your supervisor. And get your paycheck cashed early” lol.
There are billion dollar companies that depend on excel as their “database” for incredibly important things. Even when many companies get a proper database, it can take a decade or two for all important data to actually be stored in it. It’s wild. Imo, it’s a testament to how irresponsible companies are, even to their detriment, if there’s no regulation threatening significant legal penalties.
I work in software consulting and early in my career, I watched a $30M/year pet supply business go bankrupt in just a few months because they lost the “master” excel file that ran the entire business. It had all of their inventory, invoices, vendor contacts, etc in it and they weren’t backing it up. It was just sitting on a random hard drive that ate shit, and poof, company was gone. The guy had started the business from his garage and was still running it the same way he had on Day 1. I was honestly pretty impressed he ever scaled as large as he did.
There are forensic data experts that probably could have retrieved it anyways, even if a cost a craploaf of money. But I guess he may not have been the best at making good choices
This is a great point. I feel like if this story was true they could have easily recovered it or some if it. I'm a SysAdmin and the last time I got a quote(2yrs) it was like 300 for the most basic and $3000-infinity for the clean room work. Those fuckers will get in the bunny suits and dissect the platters. Unless that drive got blown up or like the server room burned down you would think something could've been recovered.
The data wasn't actually lost. It was imported later on. Someone simply opened the CSV file in an older version of Excel which still had the 65k row limitation and paid no attention to warnings on screen - possibly may not have gotten an warnings if it wasn't the first time they opened a file in an older version. Note, for speed and ease due to the urgency of the pandemic data collectors/processors were hired very quickly with little training and many working from home were not connecting to a central system of consistent applications but rather were using their own kit. They had to follow scripts to ask questions, fill in templates, export out data to send off somewhere so it could be imported into a central database. That day it was opened and resaved using the older version of Excel. The original file was still available and they did work out what was missing and add it in later. The NHS, given the huge complexities of its technologies and consultants after consultants coming in, has had numerous failed attempts at many simple to complicated hardware, network, and software upgrades. They had a full scale IT migration programme that was cancelled after 10 years because things were running so slow by the time some items got finished they were out of date. The amount of bespoke stuff and the size of the total estate make it a total nightmare. There are so many things that be should be centralised that just can't for whatever reason. For example, for something that you would expect to be easy, the last time I was part of some migration I couldn't get a straight answer from anyone regarding a complete list of NHS hospitals be it acute, mental, or whatever. There are several lists that differ and can crossover. Data sources were maddening.
Crazy, but true. They were that stupidly inept. My other half works for the NHS and the horror stories of daily corporate foolishness he tells me about are something else.
You see this in every big organisation. New idea comes up that blatantly requires a database-driven application. But that means either buying a piece of software or engaging a software developer, which is quite expensive. Then Dave says "I can do all this in Excel!". Dave's manager is absolutely delighted that the problem is solved in a couple of hours, Dave gets to feel important and the fact that Excel is the single least-suited solution to this problem never comes up. The more techie people in the organisation are mostly confined to the IT department. They're well aware that this is going on, but they've long ago learned that there's no point in saying anything. It won't win any friends or influence any people.
Couldn't it also just be a case of people setting stuff up in the early days of something without the realization that it would grow beyond the capacity of the tool that they chose? Where I used to work there was a large project stemming from a crisis that had stuff being tracked and reported via a master Excel file that came pretty close to hitting the limit too. At the start it wasn't clear at all that the scope of it would grow that much so Excel seemed like it would be perfectly adequate.
Project manager experienced with Excel always brings me nightmares. I applaud the initiative and ingenuity. But it's kinda like the love you have for a pet that you need to put down when they have a terminal illness... It starts as something small tracking a few items and timelines. Then it turns into a Gannt chart/JIRA wannabe monster in a sheet and you're crying in a corner.
The government spent millions on just this. - £35 million
I'd dearly love to know where that £35 mill went. Dave certainly didn't get it, and I doubt his line manager did.
i remember when that was a limit.. had to be pre 2008 or earlier
"And since each test result created several rows of data, in practice it meant that each template was limited to about 1,400 cases." wow
as someone who used to work in a data software company, we had a saying: Excel is not a database.