They/Them, agender-leaning scalie.

ADHD software developer with far too many hobbies/trades: AI, gamedev, webdev, programming language design, audio/video/data compression, software 3D, mass spectrometry, genomics.

Learning German (B2), Chinese (HSK 3-4ish), French (A2).

  • 0 Posts
  • 11 Comments
Joined 1 year ago
cake
Cake day: June 18th, 2023

help-circle

  • Honestly, I don’t think that there’s room for a competitor until a whole new paradigm is found. PyTorch’s community is the biggest and still growing. With their recent focus on compilation, not only are TF and Jax losing any chance at having an advantage, but the barrier to entry for new competitors is becoming much higher. Compilation takes a LOT of development time to implement, and it’s hard to ignore 50-200% performance boosts.

    Community size tends to ultimately drive open source software adoption. You can see the same with the web frameworks - in the end, most people didn’t learn React because it was the best available library, they learned it because the massive community had published so many tutorials and driven so many job adverts that it was a no-brainer to choose it over Angular, Vue, etc. Only the paradigm-shift libraries like Svelte and Htmx have had a chance at chipping away at React’s dominance.


  • The easiest way to get the basics is to search for articles, online courses, and youtube videos about the specific modules you’re interested in. Papers are written for people who are already deep in the field. You’ll get there, but they’re not the most efficient way to get up to speed. I have no experience with textbooks.

    It helps to think of PyTorch as just a fancy math library. It has some well-documented frameworky structure (nn.Module) and a few differentiation engines, but all the deep learning-specific classes/functions (Conv2d, BatchNorm1d, ReLU, etc.) are just optimized math under the hood.

    You can see the math by looking for projects that reimplement everything in numpy, e.g. picoGPT or ConvNet in NumPy.

    If you can’t get your head around the tensor operations, I suggest searching for “explainers”. Basically for every impactful module there will be a bunch of “(module) Explained” articles or videos out there, e.g. Grouped Convolution, What are Residual Connections. There are also ones for entire models, e.g. The Illustrated Transformer. Once you start googling specific modules’ explainers, you’ll find people who have made mountains of them - I suggest going through their guides and learning everything that seems relevant to what you’re working on.

    If you’re not getting an explanation of something, just google and find another one. People have done an incredible job of making this information freely accessible in many different formats. I basically learned my way from webdev to an AI career with a couple years of casually watching YouTube videos.


  • In two languages I’m learning, German and Chinese, I’ve found it to suffer from “translationese”. It’s grammatically correct, but the sentence structure and word choice feel like the answer was first written in English then translated.

    No single sentence is wrong, but overall it sounds unnatural and has none of the “flavor” of the language. That also makes it bad for learning - it avoids a lot of sentence patterns you’ll see/hear in day to day life.




  • Some minor/hard-to-notice health-related things can dramatically reduce alcohol tolerance and/or give “hangovers” shortly after starting a session.

    For me, inflammation is a big cause. I have (barely noticeable) cat allergies, and (obvious but hard to avoid) food intolerances & gut issues. If I don’t stay on top of avoiding triggers, my alcohol tolerance goes from multiple G&Ts giving a nice buzz, to 1-2 sips of G&T giving dizziness and headaches. Electrolyte imbalance can also cause it. I’ve found I have to add magnesium and potassium salt to my diet, or else I generally feel tired more, and my alcohol tolerance plummets. Once you start controlling these factors, you’ll start getting clear feedback from your body when you have too much or too little salt, in the form of water and food tasting different and general feelings of tension or tiredness.

    My advice: try antihistamines, easily-digestible meals, and/or sports drinks for a few days before you drink. If those help your tolerance, you probably have some health stuff going on - figure it out and you’ll probably find a way to generally feel better.


  • As someone with untreated ADHD, I absolutely don’t feel I’m the highest level of control in my brain. I can make all the plans and decisions I want, but I can only gently steer what I ultimately end up doing and paying attention to. My “executive function” wields ultimate power and not only can overrule me, but also prevent me from having the thoughts I want to have.

    Another indicator that I’m not the only consciousness in here: anxiety-inducing events like deadlines and exams can give me physiological symptoms even when I’ve forgotten about them. I’ll just be sitting there wondering “why is my stomach upset at me?” and only later realize it’s from stress for an upcoming test I hadn’t paid attention to.


  • Though my lizard brain demands me to be around other people, most of the things that bring me genuine life satisfaction are just easier to do solo. When I’m at purely social events I also get this sense of dread that I could be making better use of my time.

    The voice in my head is making contradictory demands, so I’ve learned to not feel bad for circumventing it. I have my own goals in life, instincts be damned.

    I find that listening to people casually talking is usually enough to satisfy the lizard brain, so I listen to a lot of stuff in the background: YouTube video essays, Twitch Just Chatting streamers, etc. When it gets particularly demanding I’ll try engaging with the people, but usually I just let my subconscious listen while I’m focusing on more important stuff.


  • ooo, I love this. It reminds me of how nice C#'s LINQ is…

    “Pipeline style” DB queries have some interesting advantages as well:

    • It’s straightforward to write efficient queries for DBs that don’t include a query optimizer stares at Datomic
    • You can split the pipeline into server-side and client-side steps when working with less capable DBs stares at most of NoSQL
    • It would be much easier to transition from a pipeline API to a non-text-based API so that our ORMs/query builders can directly talk to DBs without the overhead of generating and parsing SQL.

  • I still use Google for ~95% of my queries because I like real sources, comprehensive documentation, and not having to read a wall of text when a one-line answer would have sufficed.

    ChatGPT is a good replacement for Quora/Stack Exchange for explaining general knowledge stuff like other languages’ grammar and simple science, as well as finding authors/books/movies from descriptions when you’ve forgotten their names.

    Bard is… kinda dumb. I gave it a few chances, but it was nothing compared to ChatGPT’s free tier.