• fartsparkles@sh.itjust.works
        link
        fedilink
        arrow-up
        20
        arrow-down
        4
        ·
        1 year ago

        I feel the issue with AI models isn’t their source not being open but the actual derived model itself not being transparent and auditable. The sheer number of ML experts who cannot explain how their model produces a given result is the biggest concern and requires a completely different approach to model architecture and visualisation to solve.

        • cynar@lemmy.world
          link
          fedilink
          English
          arrow-up
          22
          arrow-down
          1
          ·
          1 year ago

          Unfortunately, the nature of the models is that it’s very difficult to get an understanding of the innards. That’s part of the point, you don’t need too. The best we can do is monitor how it’s built and what connects in and out of it.

          The open source bits let you see that it’s not passing data on without your permission. If the training data is also open source, you can get for biases e.g. 90% of faces being white males.

        • Jamie@jamie.moe
          link
          fedilink
          arrow-up
          7
          arrow-down
          1
          ·
          1 year ago

          No amount of ML expertise will let someone know how a model produced a result, exactly. Training the model from the data requires a lot of very delicate math being done uncountable times to get a model that results in something useful, and it simply isn’t possible to comprehend how the work inside is done in a meaningful way other than by doing guesswork.

        • DigitalJacobin@lemmy.ml
          link
          fedilink
          arrow-up
          3
          arrow-down
          1
          ·
          1 year ago

          We already know how these models fundamentally work. Why exactly does it matter how a model produced some result? /gen