60k of rows is nothing. Fuck, where do you find these “geniuses”?
Tbf we don’t know how many columns there are /s
The IRS just switched columns and rows. So there’s 60k rows and 330 million columns /s
deleted by creator
I do data analysis for a living, I reach out to tech and complain if I can’t open a file with a million+ lines.
There’s only one reason he wants kids and not experienced adults.
IT guy checking in.
The only time I’ve even seen drive temp sensor alarms is on server raid arrays and other similar hard drives/SSDs… Never in my life have I seen one available on a consumer device, nor have I seen any alarm for and drive temp, go off. It just doesn’t happen.
IMO, this is one of those language barriers where people call their computer chassis (and everything in it) the “hard drive”.
Applying that assumption, their updated statement is: His computer over heated.
Idk what kind of shit system he’s running on that 60k rows would cause overheating, but ok.
Wow.
I’ve been processing a couple of billion rows of data on my machine, the fans didn’t even come on. WTF are they teaching “experts” these days, or has Elmo only hired people who claim that they can “wrangle data” and say “yes” ?
He hired a bunch of 19-25 year old. Not experts
Bunch of 1337 hax0rs script kiddies who don’t understand anything but they suck elon’s balls or something idk.
These are the type of people that have deleted the French language from their GNU/Linux system.
Hey! Thats offensive to 19-25 year olds, there are many who just finished college/university and are more than aware.
They’re just role playing like in movies, with no idea of the consequences.
How on earth is it offensive to say they’re “not experts”? They’re not prodigies with PhDs. These specific young men are just technical enough and ideologically aligned.
Except they’re not, as you will know their tweet would be false after your first year of any technical (IT oriented) education.
First year? That shit is like A+ cert level knowledge or below, and A+ is damn near worthless. They would know that in the first few hours of a study guide
I was being generous when you consider the people in school who somehow pass, even when they don’t know a thing 🥲
Technical enough to be hired, is all I meant. 🙄
Even then, no. These were all obviously nepotism hires who would not have otherwise qualified.
Meh, some of them won some hackathons and scholarships, it’s pretty clear they’re otherwise at least somewhat bright but they don’t have any relevant domain knowledge.
In other words, the type of person most likely to be prone to hubris and catastrophic failures.
Apologies, if I came over as hostile. I did not get your meaning through text.
Your original comment was ambiguous as to if being an “expert” and “being 19-25” are mutually exclusive.
lol a 19-21 yo isnt going to have a degree lol,
If they went into uni straight out of high school, they could. A lot of Bachelor holders would be around that age, since they start at 18.
There is nothing wrong with being 19-25. There’s something wrong with being wholly incompetent.
There’s not really anything wrong with being incompetent, so long as you have the humility to admit it and learn from people who know better, and try not to cause harm. That’s not Musk’s minions though.
I think it’s important to differentiate incompetence from ignorance. Ignorance is not knowing. Incompetence is not being able to fulfill the requirements for your assigned task. If you cannot fulfill the requirements for your given task, then you should not be given said task.
I’ve read a story on the forbidden website where a “database” was a single table with a single column holding a single row that contained the actual data as a CSV blob. I’m willing to bet the muskies are not beyond such acts of genius.
It’s terrifying that this is plausible.
We call it New Redis!
You have to understand that the average Trump voter probably knows everything they know about computers from watching the ‘wacky-zaney hacker with personality issues/quirks’ “hack” into things by tippity tapping their fingies on a keyboard in your average copaganda performance.
This is something those types of people will believe.
You’re on the mark. I’m like Help Desk Level 2, I wouldnt even consider myself an actual wizard. The average person in my office thinks I’m Gandalf. Its scary how much these people dont know. And each one of them is out there on the internet.
Yes, his Boy Harem.
Even if querying data was processing-heavy and even if somehow the ‘hard drive’ got warm during this, then there still would need to be a hardware defect in order for the drive to overheat.
Yes, but this may be a symptom of an issue I’ve been seeing with younger programmers; they’ve siloed themselves so specifically into whatever programming they “specialize” in, that they become absolutely useless at dealing with absolutely anything else related to their job. And exasperating this issue is the fact that they’ve grown up with systems that “just work”. Windows, iOS, and android are all at the point where fucking around with hardware issues is very uncommon for the average person.
Asking this guy to solve a hardware problem is like asking hime to tune a carburetor. He likely has not the slightest clue how to start.
In my experience, a lot of software dev degree paths basically don’t even have relevant classes on hardware at all. Classes on hardware are all in IT Helpdesk and Network Admin degree paths whereas the software dev students are dumped straight into Visual Studio right off the bat with no relevant understanding of the underlying hardware or OS.
My experience does not reflect yours. Computer Architecture, Discrete Math (logic gate math), and Operating System Concepts were all required classes in my CS degree from just a few years ago.
Honestly that’s good to hear. I’ve run into some devs who are completely mystified on how to connect to a remote database and couldn’t tell a socket from sandwich.
Can I have my socket with rye. I like rye.
In my degree, we had to write kernel mods and device drivers
My CS degree had a hardware/IT support class, but A) it was entirely simulation based. We never touched any actual hardware. We “built” PC’s or identified physical issues in 3d sim software, set up RAID arrays in software, etc. B) it was super hand holdy and you only ever go over a problem once, so nothing on the class has stuck. I know much more from having built, troubleshot and maintained my own computers and network than I ever learned from that class, then learned more by doing in an actual IT support position before becoming an engineer.
I mean to be fair the sheer amount of material most university engineering programs require these days makes spending significant time on specific problems almost impossible. They try to shove so much theory into your head they lose track of practical implementation. Basically everyone I went to school with complained about the lack of practical application relative to theory, and I studied mechanical engineering which is theoretically and literally chiefly concerned with hardware.
deleted by creator
You don’t teach a farmer how an internal combustion engine works. Computers are tools to software engineers. What they need to know is how to operate them, not how to maintain them.
I’m not sure how well that analogy holds up. Farmers are usually pretty well versed in mechanical systems. To the point that now that John Deere has been screwing them over on right to repair that some farmers are even becoming versed in computer programming so they can flash the firmware on their tractors.
Farmers can build a two stroke from parts.
No, but if a farmer’s tractor is overheating (as in the gard drive conparison), I’m sure they could diagnose it.
I never said that it was impossible for a farmer to learn things outside their immediate field. Just like computer programmers often have knowledge of hardware and the general technology stack.
My point, to make it explicit to a few of the illiterates who’ve replied to my comment so far, is that it is not necessary to teach a web developer how a goddamn CPU works. They can gain nothing from that knowledge because there are at least 3 levels of abstraction between JavaScript and assembly.
And my point is that the example you used does not make the point you are trying to make, but rather the opposite. I get what you’re saying, it just doesn’t apply to farmers and mechanics.
Operating your tools and being able to maintain and repair your tools are the unequivocally essential skills for everyone in every single industry.
If you can’t, you are not a professional.
The concepts of machine logic, registers/lookups/etc are essential for every programmer. If you don’t have a clear idea about how the simplest CPU functions, you don’t have any basis of understanding the abstractions in front of you, scripting in JS. Not a professional.
no but a web dev should have some knowledge basis on what the ever living fuck their AIDs code fuelled by nothing but the cheapest source of caffeine and brain damage they have even does.
This is the entire reason why half of the internet is just broken, stupid developers who don’t know how anything works, but know how to code, making dogshit implementations of anything and everything they can get their hands on.
It doesn’t matter that the learning is segmented, you should STILL be learning about computer hardware and it’s architectural choices, it’s literally the reason why programming languages work the way that they do.
No, not really. Programming requires understanding of the underlying hardware, at least to a certain extent. Otherwise performance issues will look like dark magic and optimizing anything would be impossible.
Where do you start debugging if something goes wrong with the software and your information level is this low/ do you look at network stats? CPU utilization, paging/swapping? Is the hard disk bandwidth the bottleneck? Without at least some passable understanding of a computer architecture people like this just throw up their hands, or throw whatever tricks they know at the wall and see what sticks.
A lot of farmers are learning how they work cause the companies that sell them the equipment keep fucking them over. I would argue that farmers nowadays needs to know how that works along with basic programming to get past the anti-consumer bullshit companies put in to make it nigh impossible to fix things yourself.
doesnt matter if you know how to program, john deere is just going to put some autistic encryption and ID locking on their shit, what needs to happen is for john deere to stop fucking doing this.
Most tractors are walking computers anyway, farmers are genuinely the most multi talented people you will ever meet in your life.
the only reason farmers are afloat financially is BECAUSE they can rebuild an engine if needed.
Just look at the john deere right to repair shit. It’s literally a huge problem.
CS departments were doing poorly, but now they’re putting out farmers? No wonder all these new graduates can’t find a job.
I mean every programmer says they intend to quit and pick up farming. Might as well give them the knowledge to be successful at their late career while they’re at it
Ooh wait 'til Musk realises he can improve US agricultural efficiency.
Horseshit. Computers aren’t tools for a software engineer. Computers are tools to an administrator, an accountant. Computers are the sandbox you are building castles in as a software engineer. If you don’t understand the system upon which you build, its abilities and features, its limitations, it’s dependencies, you are going to make some stupid mistakes.
You need to understand discrete mathematics as a consequence of computer computation. You need to understand parallel processing and threading for muli-core processors. You need to understand networking, package management, security vulnerabilities, etc. from different architectures and protocols. And it ALWAYS helps to understand the very basics of a computer’s functioning, from hardware, CPU architecture, machine code, assembly/low level programming, memory management, etc.
print('Hello, World!) is day one shit for a reason. Programming language and logic is the basics. The real expertise comes from your 3rd and 4th year materials. Databases, architecture, theory of computation, discrete mathematics, networking, operating systems, compilers, etc.
computers are a tool to anybody who uses them?
If you’re using a tool, it goes without saying, you should probably have at the very least, a cursory understanding of it’s function. Lest you injure yourself gravely.
What the fuck
How is he going to fix his tractor? Wait days for John Deere to send somebody? Let the crop rot on the vine?
It is good for the programmer to know how the computer operates, as well.
You’ve never met a farmer in your life.
Just keep trying to justify your own lack of competency I guess. ¯\_(ツ)_/¯
That’s the price of specialization. Don’t ask a software engineer to troubleshoot hardware. Don’t ask a backend dev to write a frontend. Don’t ask a proctologist to look at your cough.
You simply cannot be proficient at every sub-sub-specialty. That’s why we collaborate and hand the ‘my computer gets hot’ problems to the hardware people. The alternative would be only moderately useful generalist.
I’m not asking everyone to be able to become a hardware specialist, but if you can’t even figure out “my computer gets hot” I’m not going to be able to trust anything you do. Identifying a heat issue does not take a rocket surgeon.
If it was an nvme ssd i could almost believe it. Some come with totally underspecced heatsinks
has Elmo only hired people who claim that they can “wrangle data” and say “yes” ?
There’s two issues going on:
- Elmo’s sociopathic approach to laying people off is public knowledge, and top experts have the luxury of not even applying for his jobs.
- Elmo’s ability to judge engineering talent has likely been wildly exaggerated thanks to how he has successfully bought organizations full of talented people, in the past.
60k rows is generally very usable with even wide tables in row formats.
I’ve had pandas work with 1M plus rows with 100 columns in memory just fine.
After 1M rows move on to something better like Dask, polars, spark, or literally any DB.
The first thing I’d do with whatever data they’re running into issues with is rewrite it as partitioned and sorted parquet.
My go-to tool of late is
duckdb, comes with binaries for most platforms, works out of the box, loads any number of database formats and is FAST.
From the same group that doesn’t understand joins and thinks nobody uses SQL this is hardly surprising .
Probably got an LLM running locally and asking it to get data which is then running 10 level deep sub queries to achieve what 2 inner joins would in a fraction of the time.
You’re giving this person a lot of credit. It’s probably all in the same table and this idiot is probably doing something like a for-loop over an integer range (the length of the table) where it pulls the entire table down every iteration of the loop, dumps it to a local file, and then uses plain text search or some really bad regex’s to find the data they’re looking for.
I have to admit I still have some legacy code that does that.
Then I found pandas. Life changed for the better.
Now I have lots if old code that I’ll update, “one day”.
However, even my old code, terrible as it is, does not overheat anything, and can process massively larger sets of data than 60,000 rows without any issue except poor efficiency.
Considering that is nearly exactly some of the answers I’ve received during the technical part of interviews for jr data eng, you’re probably not far off.
Shit I’ve seen solutions done up that look like that, fighting the optimiser every step (amongst other things)
I think you’re still giving them too much credit with the for loop and regex and everything. I’m thinking they exported something to Excel, got 60k rows, then tried to add a lookup formula to them. Since you know, they don’t use SQL. I’ve done ridiculous things like that in Excel, and it can get so busy that it slows down your whole computer, which I can imagine someone could interpret as their “hard drive overheating”.
They don’t understand joins? How…
Either she knows something novel, where processing data using voice coils is somehow beneficial, or is someone who calls their computer a ‘hard drive’, which summarily negates any legitimacy of technical competence.
Or wrote the code using AI without checking what it exactly does.
The AI probably used bogosort or something equivalent
I smell something, but it’s not overheating electronics.
I’ve processed over 5 million records on a laptop that’s almost 10 years old. it took two days to get my results.
there’s no way 60,000 records overheated ANYTHING.
Doesn’t actually say that 60k overheated his drive. He says that he ran a run on 60k, and that he couldn’t do the whole database due to overheating. Two unrelated statements except that 60k is the lower bound for what he could process.
Doesn’t mean he knows what he’s doing though, as pretty huge datasets are processable on quite modest hardware if you do it right.
that’s somehow worse.
a “data analyst” couldn’t cut up the work into a parallel processes and run them synchronously? what the actual fuck?
“sorry, I can only do 60k at a time.”
just fucking split them up into 6 parallel batch processes running 10k at a time. it’s fucking math, not rocket science. I’m not even an analyst and I could fucking do that much.
I don’t want to take away from the valid point of your comment, but rocket science is almost exclusively math
my hard drive overheated
So, this means they either have a local copy on disk of whatever database they’re querying, or they’re dumping a remote db to disk at some point before/during/after their query, right?
Either way, I have just one question - why?
Edit: found the thread with a more in-depth explanation elsewhere in the thread: https://xcancel.com/DataRepublican/status/1900593377370087648#m
So yeah, she’s apparently toting around an external hard drive with a copy of the “multiple terabytes” large US spending database, running queries against it, then dumping the 60k-row result set to CSV for further processing.
I’m still confused at what point the external drive overheats, even if she is doing all this in a “hot humid” hotel room that she can’t run any fans I guess because her kids were asleep?
But like, all of that just adds more questions, and doesn’t really answer the first one - why?
Have you ever heard of case of overheating hard drives within the last decade?
I don’t think I’ve seen a brand new computer in the past decade that even had a mechanical hard drive at all unless it was purpose-built for storing multiple terabytes, and 60K rows wouldn’t even take multiple gigabytes.
I have when a misconfigured spark job I was debugging was filling hard drives with tb of error logs and killing the drives.
That was a pretty weird edge case though, and I don’t think the drives were melting, plus this was closer to 10 years ago when SSD write lifetimes were crappy and we bought a bad batch of drives.
Plus, 60k is nothing. One of our customers had a database that was over 3M records before it got some maintenance. No issue with overheating lol
I run queries throughout the day that can return 8 million+ rows easily. Granted, it takes few minutes to run, but it has never caused a single issue with overheating even on slim pc’s.
This makes no fucking sense. 60k rows would return in a flash even on shitty hardware. And if it taxes anything, it’s gonna be the ram or cpu- not the hard drive.
In my experience, the only time that I’ve taxed a drive when doing a database query is either when dumping it, or with SQLite’s vacuum, which copies the whole thing.
For a pretty simple search like OP seems to be doing, the indices should have taken care of basically all the heavy lifting.
I literally work with ~750,000 line exports on the daily on my little Lenovo workbook. It gets a little cranky, especially if I have a few of those big ones open, but I have yet to witness my hard drive melting down over it. I’m not doing anything special, and I have the exact same business-economy tier setup 95% of our business uses. While I’m doing this, that little champion is also driving 4 large monitors because I’m actual scum like that. Still no hardware meltdowns after 3 years, but I’ll admit the cat likes how warm it gets.
750k lines is just for the branch specific item preferences table for one of our smaller business streams, too - FORGET what our sales record tables would look like, let alone the whole database! And when we’re talking about the entirety of the social security database, which should contain at least one line each in a table somewhere for most of the hundreds of millions of people currently living in the US, PLUS any historical records for dead people??
Your hard drive melting after 60k lines, plus the attitude that 60k lines is a lot for a major database, speaks to GLARING IT incompetence.
Pretty sure I run updates or inserts that count over 60k fairly often. No overheats. Select queries sometimes way higher.
You’ve got it all wrong, in traditional computer terminology the “hard drive” is the box that sits under the desk that collects cat fluff and cigarette tar.
/s …?
Reminds me of those 90s ads about hackers making your pc explode.
Musk gonna roll up in a wheelchair, “the attempt on my life has left me ketamine addicted and all knowing and powerful.”
I’d much sooner assume that they’re just fucking stupid and talking out of their ass tbh.
Same as Elon when he confidently told off engineers during his takeover of Twitter or gestures broadly at the Mr. Dunning Kruger himself
Wonder if it’s an SQL DB
Elon probably hired confident right wingers whose parents bought and paid their way through prestigious schools. If he hired anyone truly skilled and knowledgeable, they’d call him out on his bullshit. So the people gutting government programs and passing around private data like candy are just confidently incorrect
Even if it was local, a raspberry pi can handle a query that size.
Edit - honestly, it reeks of a knowledge level that calls the entire PC a “hard drive”.
Unless they actually mean the hard drive, and not the computer. I’ve definitely had a cheap enclosure overheat and drop out on me before when trying to seek the drive a bunch, although it’s more likely the enclosure’s own electronics overheating. Unless their query was rubbish, a simple database scan/search like that should be fast, and not demanding in the slightest. Doubly so if it’s dedicated, and not using some embedded thing like SQLite. A few dozen thousand queries should be basically nothing.
Yeah, no matter what way you disorganize 60,000 rows, the data is still going to read into memory once.
Why? Because they feel the need to have local copies of sensitive financial information because… You know… They are computer security experts.
My one question would be “How?”
What the hell are you doing that your hard drives are overheating? How do you even know it’s overheating as I’m like 90% certain hard drives (except NVMe if we’re being liberal with the meaning of hard drive) don’t even have temperature sensors?
The only conclusion I can come to is that everything he’s saying is just bullshit.
dude is 100% talking about ssds. NVME ones at that, he’s just stupid.
They have temp sensors. But have never heard of a overheating drive.
Hard drives do get hot and need some cooling but not at 60k rows. Its either made up or their computer case is made of thermal cladding
You could query 60,000 rows on a low tier smart phone. Makes no sense at all.
Can we think of any device someone might have that would struggle with 60k? Certainly an ESP32 chip could handle it fine, so most IoT devices would work…
Unless the database was designed by someone who only knows of data as that robot from Star Trek, most would be absolutely fine with 60k rows. I wouldn’t be surprised if the machine they’re using caches that much in RAM alone.
Right? There’s no part of that xeet that makes any real sense coming from a “data engineer.”
Terrifying, really.
Imo if they can’t max out their harddrive for at least 24 hours without it breaking, their computer was already broken. They just didn’t know it yet.
Any reasonable SSD would just throttle if it was getting too hot, and I’ve never heard of a HDD overheating on its own, only if there’s some external heat sources, like running it in a 60°C room
Hard Drives might do it if the enclosure is poorly designed (no ventilation), but I can’t imagine a situation where it would overheat like that that quickly, even in a sealed box. 30k is nothing in database terms, and if their query was that heavy, it would bottleneck on the CPU, and barely heat the drive at all.
As a reasonably experienced “data guy,” this seems obviously laughable, but the discussion on X is scary. This guy is a savior in the MAGA world.
We can criticize and poke fun all day, but it doesn’t matter much if our message isn’t challenging the mindset of those with other opinions.
How do we make better use of our time to impact outside opinion?
I’ve been told violence isn’t the answer and we shouldn’t just shoot nazis and nazi enablers dead.
The way most people change their mind isn’t based on facts or figures, but emotions. Specifically, in-group belonging. For most people, and this certainly includes me and you some of the time, what our in-group believes is more compelling than an out-groups supposed facts.
They see that guy as someone in their group so they believe him. They see you as a bad outside bad bad bad liar, so nothing you say is likely to get through. (This comic is worth reading on this topic: https://theoatmeal.com/comics/believe )
If you want to change someone’s mind, they have to see you as in-group. Not necessarily the same group as what you’re arguing with. We all belong to many groups. American, new yorker, white guy, middle aged, yankees fan, etc etc there are many such slices. Like how you can’t get a republican to recycle by appealing to environmental concerns (because environmentalists are out-group, so fuck them), but you might be able to get them to recycle via something like “only american ingenuity can turn trash into bridges and tanks!”
This takes a lot of time and effort, and if you don’t get them to stop hanging out with the other group, you won’t make any lasting changes.
So I think you’d need a multi prong approach:
- Get them off bad media. Facebook, fox news, etc. This is reinforcing their bad beliefs. Because they see this stuff as trustworthy in-group, it goes right into the worldview.
- Get them to stop hanging out with their shitty maga-hat friends. This is the social in-group that’s reinforcing bad beliefs.
- Get them to trust you.
- Gently introduce the idea that maybe the extreme right doesn’t have their interests at heart, etc
All of which takes a lot of time and effort, and your opposite number is basically trying to do the same thing. Except they have fox news, trump, and such in their corner.
And, again, I’m told we definitely shouldn’t just shoot extreme right wingers and other nazi sympathizers dead. Nor should we burn their houses down. If we’re an emergency responder, we definitely shouldn’t let them die while thinking to ourselves “they would let so many die. without a thought, their passing deserves no mourning” or similar.
You should definitely nullify if you’re on a jury and someone allegedly did violence to a shitty ceo or red-hat, though, bu that’s getting off topic.
Wow, that was an awesome rabbit hole, thank you for the link.
If you want to change someone’s mind, they have to see you as in-group.
Maybe a less manipulative-sounding way to phrase that might be that we should remind people that we’re all in it together. The far right media and their billionaire buddies have spent the past decade and a half dividing us, and they succeeded. Idk what it would take to unite this country again, but it at least is a little comforting to have a clear problem statement.
I’ve been thinking about this a lot lately. I’m from the UK, and whilst things are less politically dire here than the US, it’s still pretty grim. Both the Conservatives and Labour seem reluctant to actually meaningfully tax the rich, even as the working class (and to a lesser extent, the middle class) are being squeezed by a cost of living crisis and general hopelessness. Parties like Reform are taking the racist “things are bad because we have too many immigrants” and I’ve recently realised that I need to stop resenting people for being taken in by that rhetoric; people are desperate and there aren’t people in the mainstream pushing for alternatives (besides Reform). These people have a lot in common with me, such as recognising that we’re being fucked but the system, but we just disagree on the solution. It’s hard, but ultimately necessary to be able to be in solidarity with people like Reform’ voters
I’ve been told violence isn’t the answer
By the very same people asking the question: What are you gonna do about it?
Where ‘it’ is your oppression.
We must make better memes
I’m not even joking, the world runs on memes now. It’s fucking stupid, but we must shitpost to save ourselves
I agree some form of consistent opposition messaging is needed.
The maga world talks in consistent themes and terminology, which creates a psychological advantage. Unfortunately, it’s playground psychology, but if that’s the game being played you need to find a way to win at it.
I can’t remember the particular phrase that was used, but I heard an argument recently that we need to be more like politicians going on an interview and ensure that we’re more on message. For example, it’s fairly obvious by now that economically, the problem is wealth inequality, but I see fairly surprisingly few people discussing that.
Guy
That’s probably who I’m remembering; I recently discovered his work.
Compelling point. I just found that arguing with „these kind of people“ (livibg in europe, so no MAGA‘s here but like-minded, conservative fundamentalists etc.) leads to nowhere. It‘s kind of like the covid-conversations. And often I heard „you can‘t make them change their minds, so just let them be“. Still, I think this behaviour leads to isolation and separates us as a people even more.
Long story short: good question. If you found the answer, let me know.
I can only speak from my very limited experience. My father is the very example of a person who has some beliefs and tries to judge whole world through those beliefs. Everyone who doesn’t share those beliefs is an enemy. If you don’t believe in extreme opinion A, you automatically have to believe in extreme opinion B which is the opposite of A. You probably know such people.
Over last years, we yelled at each other lots of times, but that lead to nowhere. What actually helped was that finding the common ground. To make him understand that just because I don’t agree with his side, it doesn’t necessarily mean that I’m shilling for the other one. Not everything is bipolar. From my father’s perspective, everyone has to be either pro-Russian or pro-American (which is funny from today’s perspective, but I guess you get my point). You point out that Russia did something bad? He will tell you “Yeah and USA did <something>! You don’t have an issue with that”. And that’s the thing. To make him understand, that I DO have an issue with that. World is not a football match where you have to take a side and fully commit to it. You don’t have to go “full in” on a topic. Your opinion can be nuanced based on the actual topic, not just dumbed down into “my side thinks that A, so I agree with A. Your side thinks that B, so you have to agree with B”.
Before that I never had much success in having a proper discussion. It always ended up in a screaming match, because he wasn’t listening to arguments. He simply knew, that I had the “other” opinion, so my opinion was automatically wrong. Now he knows, that I don’t fully agree with anyone. He now somehow understands that my opinions are based on a set of principles, not on a tribalistic “my team” vs “your team”. And by understanding that, he’s more open to actually having a discussion on a topic, not just trying to convert me from “bad side” to “good side”.
And don’t get me wrong. He still believes in what he believes in. But he’s more open to accepting that not everything “his side” says is automatically correct. And that by itself is a small victory for me.
This shit sounds like when your mom tells you that the Facebook printed out her bank statement on the fax machine. I’m not smart enough to even guess how you did something dumb enough to make that happen.
How bad are you at writing queries? How does your hard drive overheat even under 100% load? Do you have it smothered under a blanket? Did you crack it up and expose it to cheeto dust? What does running a query on your, presumably, remote database even have to do with your harddrive in the first place? Are you trying to copy the entire database locally to a laptop? Do you know how to tie your shoes yet, or are you still on the velcro?
This shit sounds like when you’re mom tells you that the Facebook printed out her bank statement on the tax machine.
My dear sweet mother asked me somewhere around 2005-06 “If we can fax paper, why not groceries, or pizza delivery?”
Apparently she had believed, for decades, that fax machines literally transported physical paper over phone lines. She has a college degree, and my family is wealthy.
Do not underestimate the mind boggling technical and scientific ignorance of old people who should know better.
🤣
I remember thinking that when I was a kid and saw a clip on PBS Kids of a kid using their home fax machine to fax their dad a picture they drew at work. I was also like 4 so teleportation was still distinctly possible to my brain
A laptop should easily handle a database of 60,000 rows. I run much bigger databases on my own laptop for development purposes.
Not if it is filled to the brim with 3 TB of porn
Delete Ass Master volume 7 to make room - that one wasn’t any good anyways.
Really went downhill after volume 3. You hope that they’ll get back on course by volume 6 at least, but alas.
I can personally vouch for that not causing a problem either.
This sounds like trying to do stuff in Excel? The computer isn’t overheating but the amount of memory needed is very high which would make it run poorly. They might interpret that as overheating?
It also makes sense if they are on calling the entire computer “the hard drive” like grandma and the fans kicked on.
Yeah, everyone commenting about being able to handle billions of rows easily, which obviously very true if you are worming with sql or similar.
But this is probably some finance kid, investment banker analyst, and only knows how to use Excel.
60,000 rows in excel with formulas, if not done efficiently, can for sure make you computer a little toasty.
Could be a gen 5 nvme drive without adequate cooling. Them bastards can run hot. Especially the early gen 5 drives.
But with only 60k rows would one even have enough time to overheat?
Literally every time someone dismisses Wikipedia, it’s because they believe something crazy that Wikipedia told them is wrong.
I checked conservapedia once, and its actually unhinged. If someone tells you to look at that, or reccommends it, they’re crazy.
YES . its so unhinged . they have an entire page discussing if Obama is actually a Muslim
Flashback to 2014.
Did they ever finish their own bible translation? The one they started because King James was too woke.
“I read a book with a typo once. Libraries are a scam.”
Libraries are a scam if they weren’t they’d still have VHS rentals.
God they picked out the ONE possible thing they could criticize her for, there’s like 3 other things RIGHT NEXT TO THAT
“Software Engineer” was literally right next to it.
She is vehemently against crypto. She runs the newsletter “web3 is going great” - https://www.web3isgoinggreat.com/
heh
Molly White is very bright, and she makes them feel inadequate so they “have to” attack her. It’s truly pathetic.
But her last name is White so it’s a real dilemma for them.
“YOU’RE JUST JEALOUS” is such a fucking pussy-ass response, too.
Hard drive was made by Tesla
I used to perform data analysis of robotics firmware logs which would generate several million log lines per hour and that was my second job out of college.
I don’t know how you fuck up 60k lines that bad. Is he nesting 150 for loops and loading a copy of the data set in each one while mining crypto??
I would love to see the output of EXPLAIN on his query to see what he’s doing.
Substring searches in unindexed large string columns or cartesian explosion caused by shitty joins would be my initial guess.
Largely ignorant, but data-curious person here.
…what?
Storing large volumes of a text in a database column without optimization, then searching for small strings within it. It causes the database to basically search character by character to find a match by reading everything from disk. If you use indexes the database can do a lot of really incredible optimization to make finding values mich faster, and honestly string searching is better suited to a non-relational DB engine (which is why search engines don’t use relational DBs).
Cartesian explosion is where you join related data together in a way that causes your result set to be wayyyy bigger than you expect. For example if you try to search through blog posts, but then also decide to bring in comments to search, then bring in the authors of those comments and all their comments from other posts. Result sets start to grow exponentially in that way, so maybe if you only search a few thousand blog posts you might be searching through millions of records because you designed your queries poorly.
If there’s something you want to search by in a database, you should index it.
Indexing will create an ordered data structure that will allow much faster queries. If you were looking for the username gazter in an unindexed column, it would have to check literally every username entry. In a table of 1000000 entries it would check 1000000 times.
In an indexed column it might do something like ask to be pointed to every name beginning with “g”, then of those ask to be pointed to every name with the second letter “a” and so on. It would find out where in the database gazter is by checking only six times.
Substring matching is much more computationally difficult as it has to pull out each potentially matching value and run it through a function that checks if gazter exists somewhere in that value. Basically if you find yourself doing it you need to come up with a better plan.
Cartesian explosion would be when your query ends up doing a shit load of redundant work. Like if the query to load this thread were to look up all the posters here, get all their posts, get the threads from those posts and filter on the thread id.
That’s very clear, thanks.
I’m guessing you’d have to search the database to make the index, right? To search for ‘gazter’ you’d have had to go over the whole dataset and assigned each entry with a starting letter value, and so on?
When it comes to searching the database, the index will have already been created. When you create an index, it might take a while as the database engine reads all the data and creates a structure to shadow it. Each engine is probably different and I don’t know if any work exactly like that, but it’s an intuitive way to understand the basics of how B-trees work. You don’t really need to think much about how it works, just that if you want to use a column as a filter, you want to index it.
However, when you’re thinking about the structure of a database it’s a good idea to think what you’ll want to do with it before hand and how you’ll structure queries. Sometimes searching columns without an index is unavoidable and then you’ve got to come up with other tricks to speed up your search. Like your doctor might find you (i’m presuming gaz is sort for gary and/or gareth here) with a query like
SELECT * FROM patients WHERE birthdate = "01-01-1980" AND firstname LIKE "gar%"The db engine will first filter by birthdate which will massively reduce the amount of times it has to do the more intensive LIKE operation.
Skill issue, as the kids like to say.
What is this, a table for ants? Because that’s the average number of ants in an ant colony and it’s nowhere near an impressive amount of rows to be doing any sort of processing on. It wouldn’t be an impressive amount of rows if your rig was an i386DX-33 running off a 5” floppy.
Exactly, 60k rows is negligible enough in most cases that you can just treat it as free unless you’re doing a cross join on it or something, unless he’s doing something like using an unordered text file as his database with no ram or cache
Even then, I can crank through orders of magnitude more data on my craptop
Buddy’s probably running code he got from GitHub Copilot that is used to do a visualization of a bubble sort for learning purposes.
