Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

50GB is a really small data set.


Eh, by what measure? Realistically it's probably bigger than 90% of all Mongo datasets.

It's tiny if you're a massive company and it's massive if you're a tiny startup.


50GB easily fits in RAM. It's a small dataset.


If you can run the dataset comfortably on a Macbook then it's very, very small.

Heck, you can even just use grep over 50GB reading straight from disk. It's tiny.


Is an argument based on the premise that relative terms have absolute meanings a good use of people's time here?


A recent work Slack chat had a dev asking what a particular table contained. They were going through our data inventory and found a randomly-named table 18TB in size. When I ran "select count()" against it, I got back 5,325,451,020,708 rows (that's a copy-and-paste).

50GB isn't trivial, but it's utterly manageable.


It seems a bit wrong if you have a 18TB table but no idea what it contains...


It was a temp table that we hadn't garbage collected yet. We don't make a habit of leaving that much junk data around, but it bumped our monthly storage bill several percent, not like tripled it.


Was this a relational or NoSQL DB?


It's primarily in things like Spark and Snowflake that act like relational DBs as long as you squint the right way.


in my experience it qualifies as "medium"


If it can be stuck in a sqlite database and run on a developer laptop, then no, it is not medium by any standard.

Please elaborate why you think 50Gb is anything other than a small dataset that can fit in memory on any half-decent server though.


[edit] in the spirit of not being a condescending tool to you, i'll replace my original reply with this: https://en.wikipedia.org/wiki/Long_tail


I'm assuming this is a joke. You can run databases that size without any of the fancy scalability stuff - no sharding no anything. I'd actually recommend that, it's makes admin super easy!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: