this post was submitted on 31 Oct 2023
49 points (87.7% liked)

Technology

59428 readers
3118 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

Breakthrough Technique: Meta-learning for Compositionality

Original :
https://www.nature.com/articles/s41586-023-06668-3

Vulgarization :
https://scitechdaily.com/the-future-of-machine-learning-a-new-breakthrough-technique/

How MLC Works
In exploring the possibility of bolstering compositional learning in neural networks, the researchers created MLC, a novel learning procedure in which a neural network is continuously updated to improve its skills over a series of episodes. In an episode, MLC receives a new word and is asked to use it compositionally—for instance, to take the word “jump” and then create new word combinations, such as “jump twice” or “jump around right twice.” MLC then receives a new episode that features a different word, and so on, each time improving the network’s compositional skills.

you are viewing a single comment's thread
view the rest of the comments
[–] A_A@lemmy.world 9 points 1 year ago* (last edited 1 year ago) (3 children)

Edit : Please read @DigitalMus@feddit.dk's comment before mine.


Hey folks, I believe this is really big.

Traditional deep neural network's training requires millions of example and so, despite its great success, is immensely inefficient.

Now what if learning of these machines was as fast or faster than a human's ? Well, it seems this is it.

Look at how large language models are disruptive for many sectors of society. This new technology could accelerate the process exponentially.

[–] TropicalDingdong@lemmy.world 3 points 1 year ago (1 children)

Traditional deep neural network’s training requires millions of example and so, despite its great success, is immensely inefficient.

Is this a limited advancement in training techniques? Right now I'm working on several types of image classification models. How would this be able to help me?

[–] A_A@lemmy.world 2 points 1 year ago (2 children)

Sorry, I just read a lot... but I don't work in this field.

[–] ripcord@kbin.social 2 points 1 year ago (2 children)

If you don't really understand it, why do you believe this is so big?

[–] A_A@lemmy.world 1 points 1 year ago* (last edited 1 year ago) (1 children)

I am not sure what "image classification models" incompasses. I would have to read more and understand and I don't have enough time and energy.
Yet in the past I have read and understand a few books about neural networks and this new article in nature is something else : it's clear when reading it.
( also to @TropicalDingdong@lemmy.world )

[–] TropicalDingdong@lemmy.world 1 points 1 year ago

I mean is this any different than standard gradient descent with something like Adam as optimiser.

That's my assumption based on the headline. But the quick skim I gave the article seemed to only discuss it in the context of NLP. Not exactly my field of study.

[–] QueriesQueried@sh.itjust.works 1 points 1 year ago* (last edited 1 year ago)

~~Admittedly, they were quoting someone else in the message you responded to. That may have been edited after the fact, but the person they're quoting did in fact say those words ("this is big").~~

It was I who couldn't read, as that is not what happened.

[–] TropicalDingdong@lemmy.world -2 points 1 year ago

Based on your reading, swag?

[–] registrert@lemmy.sambands.net 2 points 1 year ago (2 children)

It's over for us useless eaters, no matter how useful one is we will always be useless compared to what's coming.

[–] TropicalDingdong@lemmy.world 4 points 1 year ago

It’s over for us useless eaters

Look at this guy over here sucking up all that oxygen.

[–] A_A@lemmy.world 1 points 1 year ago* (last edited 1 year ago)

There is still some hope ; maybe the machines will have more compassion than humans do, or maybe we are inside the matrix already as useful parts.
There are so many unknowns in the future and our insights are so limited.

[–] ExLisper@linux.community 1 points 1 year ago (1 children)

Now what if learning of these machines was as fast or faster than a human’s ?

What do you mean? It's already faster than human's. I takes years for a person to learn basic language and decades to gain expert knowledge in any field.

[–] A_A@lemmy.world 1 points 1 year ago (1 children)

What is meant here (and said as such in the article) is that humans can learn from a single example while deep neural networks takes thousands or millions (of examples) to learn.

[–] ExLisper@linux.community 1 points 1 year ago (1 children)

Ok, but neural networks can process way more examples per second so 'faster' is not really the right term here.

[–] A_A@lemmy.world 1 points 1 year ago

Yes you are right. And I was hoping for someone more knowledgeable to help clarify this topic.

Well I was lucky with the comment of @DigitalMus in here, if you would like to read it.