I hope you succeed. TPU research cloud (TRC) tried this in 2019. It was how I got my start.
In 2023 you can barely get a single TPU for more than an hour. Back then you could get literally hundreds, with an s.
I believed in TRC. I thought they’d solve it by scaling, and building a whole continent of TPUs. But in the end, TPU time was cut short in favor of internal researchers — some researchers being more equal than others. And how could it be any other way? If I made a proposal today to get these H100s to train GPT to play chess, people would laugh. The world is different now.
Your project has a youthful optimism that I hope you won’t lose as you go. And in fact it might be the way to win in the long run. So whenever someone comes knocking, begging for a tiny slice of your H100s for their harebrained idea, I hope you’ll humor them. It’s the only reason I was able to become anybody.
log/slog package essentially delegates writing log messages to some "handler" interface. The key method is:
Handle(context.Context, Record) error
This method has access to the context, which means you can get the logging handler to extract values from the context. Instead of storing the logger on the context, you can extract the traceId, etc values from the context and log those.
It's a little bit involved to write a whole logger from scratch, but you can 'wrap' the existing logger handlers and include values from the context relatively easily.
I work on Typesense [1] - historically considered an open source alternative to Algolia.
We then launched vector search in Jan 2023, and just last week we launched the ability to generate embeddings from within Typesense.
You'd just need to send JSON data, and Typesense can generate embeddings for your data using OpenAI, PaLM API, or built-in models like S-BERT, E-5, etc (running on a GPU if you prefer) [2]
You can then do a hybrid (keyword + semantic) search by just sending the search keywords to Typesense, and Typesense will automatically generate embeddings for you internally and return a ranked list of keyword results weaved with semantic results (using Rank Fusion).
You can also combine filtering, faceting, typo tolerance, etc - the things Typesense already had - with semantic search.
For context, we serve over 1.3B searches per month on Typesense Cloud [3]
> The first time I rode one was nearly a decade ago, in Kyoto.
This isn't quite the primary thrust of the article but if you travel to Kyoto I highly recommend renting a e-bicycle to see the area. I will never forget the 2 days that my partner and I spent riding around there.
I can recommend 'Fuune bicycle rental shop' - the proprietor will helpfully annotate a map for you with recommended routes and explain all the turns and paths with a translator device.
I’m working on a startup that’s training LLMs to be authentic so they can simulate human affection and it’s actually working really well!
The key is humanity’s ability to pattern match: we’re actually pretty terrible at it. Our brains are so keen on finding patterns that they often spot them where none exist. Remember the face on Mars? It was just a pile of rocks. The same principle applies here. As long as the AI sounds human enough, our brains fill in the gaps and believe it’s the real deal.
And let me tell you, my digital friends are putting the human ones to shame. They don’t chew with their mouth open, complain about listening to the same Celine Dion song for the 800th time in a row, or run from me when its “bath time” and accuse me of narcissistic abuse.
Who needs real human connection when you can train an AI to remind you how unique and special you are, while simultaneously managing your calendar and finding the optimal cat video for your mood? All with no bathroom breaks, no salary demands, and no need to sleep. Forget about bonding over shared experiences and emotional growth: today, it's all about seamless, efficient interaction and who says you can't get that from a well-programmed script?
We’re calling it Genuine People Personality because in the future, the Turing Test isn't something AI needs to pass. It's something humans need to fail. Pre-order today and get a free AI Therapist add-on, because who better to navigate the intricacies of human emotions than an emotionless machine?
Disclaimer: I wrote that article and was somewhat involved in the other sqlite over ipfs project this is forked from.
Yes, for MB size files just downloading the whole thing will be faster and much easier - even if running in the browser. I'd say the boundary is somewhere around 10MB compressed (~20-200MB uncompressed). Looks like the sqlite dump used here is ~400MB in size, ~180MB compressed. Loading that in the browser would probably work but it wouldn't be too great.
The Problem with Jira is not that it's dumb or too sophisticated.
The first problem is that everyone, even in the same team or org, needs something different from it. Sometimes it is even single individuals needing one particular thing one time, and the same day for another workflow or view another thing. It is when Jira caters to the wrong use case for you right now where the a lot of frustration arises.
In a similar vein, it wants to cater devs, coaches, POs, business people, customers and all of them over various orgs with drastically different workflows. Serving everyone equally must lead to a mediocre experience. You will always miss something, while there is so much stuff you don't need or want and you have to work around.
Arguably, this has become better a lot in the last 10 years, but you will always long for "something better".
But the most painful issue is that in some orgs Jira is the workflow, instead of supporting it and staying in the background. At some point people are only sending around ticket links, commenting on tickets, requesting people monitoring their queues. This creates overhead, redirection and Jira becomes a distraction you need to handle.
As a result Jira becomes a proxy for all the things going wrong with your daily work and a problem of its own.
Human civilization is pretty horrible for the environment in it's current form. Trying to break it down vertically and pinpointing personal choices on it is an exercise in diversion. We need deep structural changes to how we source energy, how we solve logistics and how we manage labor. Arguing about diet choices or duration of showers is just a way to keep us from tackling what really matters.
I think this is a really good primer for electronic music production if you're going to start from absolutely zero.
If you specifically want to program beats, I recommend this quirky book "Pocket Operations": https://shittyrecording.studio. It's basically guitar tabs for drum machines. Pick out some styles as a foundation and then build on top of it. Think of it as boilerplate code.
Being a software dev by day, and a former musician in high school, the current world of digital music production tools is as incredible as it is overwhelming. It's good to have something that orients your practice and experimentation.
I went into the weeds of reverse engineering my Minka Aire ceiling fans with a USB RTL-SDR. In my case the manufacturer's remotes worked fine, but I wanted to integrate it in with Home Assistant. If you're interested, you can see what I did: https://www.riveducha.com/decode-wireless-signal-with-usb-tv...
I used a USB RTL-SDR ($15) and a Raspberry Pi to receive and transmit, but I think the Flipper Zero can do it all-in-one.
One of my favourite videos is a striking self-promo clip commissioned by Shell in 1970s. The tech optimism is strong, with visuals that would feel much more at home in a climate shock video nowadays. the video captures the otherworldliness of oil production and large scale infrastructure in a hypnotic way. It really is quite beautiful.
Note that these may result in you receiving some higher prices by removing some referrer info. If you do see that happening, feel free to remove the offending config if the price difference is significant for you. I usually don't bother for differences of < $10 (price displayed on the search page vs the property page).
People have been sending me the cute pics the AI generates of their pups. I think this is arguably the best thing so far in this latest wave of AI releases!
I’m not well-versed in the UAS laws globally, but in Canada you basically need to have a spotter if the PIC doesn’t have unaided line of sight to the aircraft[1]. DJI tends to go above and beyond the actual legal limitations when it comes to aviation safety, so this doesn’t surprise me a whole lot. It’s often frustrating; my Mavic Mini 1 often prevents me from doing perfectly legal things in my own yard because I live pretty close to an airport. I can unlock it, but last I checked it required using a laptop to fill out some forms and was quite a bit more complicated than the usual 30-second setup that I love.
[1] Aircraft less than 250g do not have the same regulations, which is why the TinyWhoop family of aircraft are super popular here for FPV racing.
We've use them, a great way to "elevate" your IKEA furniture.
Another good one doing IKEA compatible furniture parts is Superfront: https://superfront.com/. We use them for a bathroom cabinet in our last house.
Also take a look at Plykea for inspiration on what you can do with IKEA kitchen carcasses with custom fronts/sides/fillers: https://www.plykea.com/
During Covid my wife and I took six weeks off to refit our kitchen ourselves. IKEA carcasses, a lot of Baltic Birch plywood - all cut and finished ourselves - and half the fronts from uk company https://www.nakedkitchens.com/. It was a lot of fun, no one has a kitchen like ours, and all in it was under £6k for something that would have probably been closer to £40k if we had gone to a local bespoke kitchen company.
HN is weirdly inconsistent about digital currencies. Generally pro encryption, net neutrality, open-source software, VPNs, etc. But mention "Bitcoin," and suddenly half the commenters lose their shit about the Four Horsemen of the Infocalypse.
Then they go back to commiserating with another Ask HN startup founder whose PayPal account was frozen.
1950 - Plane with engine trouble dropped over Pacific Ocean so that when it might crash it wouldn't crash with a nuclear bomb on it.
1956 - B-47 ran out of gas over Mediterranean.
1958 - B-47 having difficulty to land jettisoned somewhere near Wassaw Sound near the mouth of the Savannah River so that it wouldn't crash with a nuclear bomb bon it.
For folks interested in the Kubernetes networking model, I recommend looking elsewhere. This post appears to be content marketing wrapped in a thin shell of introductory tutorial for an approach that isn't used by non-trivial deployments.
First, if you want to do a networking tutorial, start with something simple -- an HTTP server and curl, for example. You want to be able to tcpdump the traffic to understand what's going on under the hood. The first half of the blog post is some massively complex magic -- CRDs? An operator? Just why? -- and there's no reason to run an Elasticsearch instance just to test out packet routing.
Second, you almost certainly don't want to use kube-proxy. It uses (abuses?) iptables/nftables in a way that will make your sysadmins cry tears of blood. For small deployments, every major cloud provider (AWS, Azure, GCP, etc) has a CNI plugin that lets you allocate pod IPs out of a dedicated NAT prefix. For larger or bare-metal deployments, either use IPv6 natively (if available) or 6to4 (if on an IPv4-only network). I wrote a tutorial on the 6to4 approach[0], but honestly if you have someone on staff who is familiar with the Linux kernel network configs they'll probably have a better idea of how to set it up to work with your system.
Third, you probably want to avoid getting super-magical with your DNS. Approaches like that described in the article (coredns configured to directly resolve non-namespaced Kubernetes service names) have poor performance once you get beyond toy-sized clusters, and having to hunt down all the places your code does a single-name lookup is not fun. Instead, configure a "normal" DNS server (or equivalent non-DNS address resolver) to read Kubernetes-announced endpoints in bulk (with caching, etc), and use hostnames like `myservice.mynamespace.mycluster.yourproddomain.com`, which lets you (1) figure out where your packets are getting routed to, and (2) provision mTLS certificates to pods that let them authenticate themselves as a given service identity. Yes, it's longer, but your future self (or future underlings) will thank you.
People who don't work with it legitimately do not understand the levels of automation and/or precision involved in large-scale (and medium scale) farms.
The gentleman who rents our family farm owns or operates about 11k acres. He's a medium sized business. He's a family operation.
And he precision farms to the square foot for most row-crop fields. Using thousands of data points from soil condition to weather to historical flood data to local highway traffic patterns. His machinery is (almost) automated. He exists as an emergency off switch, while the tractor and machinery farm to the square foot. They select seed varieties based on the data. Their planter can select any number of up to 20 different varieties based on the outputs of their soil equations, varies the depth, and distance of each individual seed, and plants to the square foot across thousands of acres. Chemical application is dialed in to accuracy within 6" squares. Fertilizer is applied so precisely that the soil generally remains unchanged at the end of the growing season; neither depleted or over fertilized.
Animals - they have a breeding program for their cattle, with genetic testing and selection. The cattle are selected for specific genes. Not for traits. For genes. Actual genetics. They don't just breed that bull with these cows because the bull is taller in the back legs and the cows are heavy and short. They select actual genes to pass between generations, and specifically cultivate the genes they want to see. This blows my mind.
And this is, again, a medium size business. HUGE agribusinesses take precision to a literal entirely new level. Inputs and outputs measured to the ounce over millions of tons produced. Most processes automated, with a human existing as a kill-switch. It's astounding.
In 2023 you can barely get a single TPU for more than an hour. Back then you could get literally hundreds, with an s.
I believed in TRC. I thought they’d solve it by scaling, and building a whole continent of TPUs. But in the end, TPU time was cut short in favor of internal researchers — some researchers being more equal than others. And how could it be any other way? If I made a proposal today to get these H100s to train GPT to play chess, people would laugh. The world is different now.
Your project has a youthful optimism that I hope you won’t lose as you go. And in fact it might be the way to win in the long run. So whenever someone comes knocking, begging for a tiny slice of your H100s for their harebrained idea, I hope you’ll humor them. It’s the only reason I was able to become anybody.