Skip to content
TheChatArchiveThe Chat Archive
gpt-4-0314· OpenAIAi2 WildChat-4.8M (AI2)

Summarize this and list important information as bullet points: Skip to content…

  1. User

    Summarize this and list important information as bullet points: Skip to content

    Advertisement Menu

    SearchSearch Search... Published on May 23, 2023 In Endless Origins Meta Finally Cracks the LLM Code Without RLHF So far, Meta has been reluctant with LLMs and chatbots, but is now rushing into the open-source without reinforcement learning with human feedback (RLHF) By Mohit Pandey

    Meta Builds a Moat in AI

    Listen to this story Meta is back in the LLM business. Google had claimed that neither they nor does OpenAI have a moat when it comes to AI, but also acknowledged that open source will pave the way for the future of generative AI. The interesting thing about this is Meta is being touted as the leader for open-source generative AI by developers because of its open-source LLM, LLaMa.

    Meta has released another open-source model called MMS: Massively Multilingual Speech. The model is made for speech-to-text and text-to-speech in 1100 languages with the ability to recognise 4,000 spoken languages. French computer scientist Yann LeCun said that the model has half the error rate of OpenAI’s Whisper. This is a huge advancement when it comes to multilingual speech AI technology.

    Besides this, yesterday, LeCun shared Meta AI’s latest breakthrough in LLMs. LIMA, made in collaboration with Carnegie Mellon University, University of Southern California, and Tel Aviv University, is a 65 billion parameter model built with LLaMA and fine-tuned with a standard supervised loss with only 1,000 carefully curated prompts and responses.

    The most interesting part is that it does not require RLHF (reinforcement learning with human feedback), which is being implemented within ChatGPT and Bard, and told as the differentiating and the most important factor for its working. There is a possibility that this model might be implementing self-supervised learning, which LeCun has been the biggest proponent of all this while.

    When it comes to performance, LeCun claims that this LIMA is on par with GPT-4 and Bard. People have been critical of this. And though excited about the development, the community is also calling out that the benchmarks to compare it with GPT-4 and Bard are not accurate. The paper is still under review.

    Recovering Doomer LeCun said in an interview, “The platform that will win will be an open one. Progress is faster when it is open.” This might seem a little contrary to the stance that the company took during this AI chatbot race. Before releasing LLaMa in February, the company was keeping itself away from LLMs. LeCun used to call these technologies “great but nothing revolutionary” and thus was also sceptical about releasing such technology to the public.

    A lot of it is because the company had actually ventured into the field before any of the competition. BlenderBot and Galactica, two separate chatbots were released by the company but had to be shut down when they turned into hallucinating disasters. Probably, a lot of cynicism towards this LLM-based technology was just trauma from the past.

    Just before the release of LLaMa, Mark Zuckerberg had made a Facebook post talking about reorganising its teams’ structure and focusing on generative AI technology. But, instead of building a chatbot to compete with Microsoft, OpenAI, and Google, the company took an interesting turn, and gave away its most precious asset to open source.

    The still-under-review paper of LIMA shows the possibility. There is a chance that the company might venture into the field of chatbots, again. The company had been reluctant to release its models to the public earlier because of their capability of generating misinformation and toxic content, as explained by LeCun.

    Now the company has taken a different approach and released its model to the general public that also includes weights, making it a lot easier to make chatbots for developers. Seems like Meta wants a thousand chatbots to exist, but won’t make one.

    Is LLaMa the moat for Meta? The tables have turned. Google and OpenAI, the companies which were in a rush to make their chatbots public, even though keeping the technology behind closed doors for most of the cases, are now considering regulations around the technology. Meanwhile, Meta, which was always not so sure about the technology, is now making their most capable models open source.

    Seems like Meta is banking on the open-source community instead of trying to build a chatbot like Google or OpenAI. For Meta, the community that is building models with LLaMa is the moat. Just like developers and creators flock around StabilityAI for its open source software, people are building model after model over LLaMa, that are outperforming GPT-4 or Bard on less computation requirements as well.

    Google’s leaked document said, “The modern internet runs on open source for a reason and we should not be expected to catch up with it.” Even though Google, Microsoft, and OpenAI’s heads were called to the White House to discuss regulations around AI, Meta was left out, citing the reason that the meeting “was focused on companies currently leading the space”.

    Even though Meta does not have a customer-facing AI product, they are not too behind on it as well. According to reports, Zuckerberg, during April’s earnings call, mentioned ‘AI’ at least 27 times, which might be less compared to Sundar Pichai chanting it 140 times at Google I/O, but still the company has been integrating generative AI into a lot of its apps, similar to Google.

    Moreover, the FAIR department of the company has been publishing papers at a rate faster than OpenAI or Google, both of which are now more concerned about protecting their technologies.

    It might be too early to tell if Meta taking sides with the open-source community would actually make a mark for them in the AI arms race. But there is no doubt that LLaMa has indeed made an impact on Google and OpenAI. Sam Altman has already discussed imposing restrictions around the technology with the US Senate.

    Download our Mobile App

    Mohit Pandey Mohit dives deep into the AI world to bring out information in simple, explainable, and sometimes funny words. He also holds a keen interest in photography, filmmaking, and the gaming industry.

    Subscribe to our newsletter Join our editors every weekday evening as they steer you through the most significant news of the day. Email Enter your email Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions. Our Upcoming Events Deep Learning DevCon 2023 May 27, 2023 | Online Register Rakuten Product Conference 2023 31st May - 1st Jun '23 | Online Register MachineCon 2023 India Jun 23, 2023 | Bangalore Register MachineCon 2023 USA Jul 21, 2023 | New York More Details Cypher 2023 Oct 11-13, 2023 | Bangalore Register 3 Ways to Join our Community Telegram group Discover special offers, top stories, upcoming events, and more.

    JOIN TELEGRAM Discord Server Stay Connected with a larger ecosystem of data science and ML Professionals

    JOIN DISCORD COMMUNITY Subscribe to our Daily newsletter Get our daily awesome stories & videos in your inbox SUBSCRIBE MOST POPULAR

    Council Post: Towards An Ethical Tech Revolution: Building Responsible AI Practices A responsible AI framework enables companies to track and mitigate bias and create transparent and explainable AI models, prevent misuse and adverse effects of AI, determine who to be held responsible if something goes wrong, and ensure compliance with security, privacy, and associated regulations.

    Generative AI in Gaming is a Double-edged Sword Twelve years after its release, The Elder Scrolls V: Skyrim is being reinvented with the help of generative AI

    How Splunk Can Save the Day for Enterprise AI “The use of AI could raise compliance and privacy concerns, particularly in industries such as finance and healthcare”

    Has OpenAI Lost the Open Race? Will OpenAI go back to its initial roots of being ‘open’ with a proposed plan to release an open-source model?

    Google Goes Silent on Quantum Computing Project. What’s Cooking? Google’s ‘bold and responsible AI’ approach might lead to their fall.

    LLM Wars: PaLM 2 VS GPT-4 Google claims that PaLM 2 demonstrates enhanced reasoning capabilities compared to GPT-4, particularly in tasks like WinoGrande and DROP, with slight advantage in ARC-C as well.

    How this Company Utilises Generative AI to Transform Online Branding SingleInterface is currently using LLMs to try to replicate the personalities of a sales agent and solving the problem of customers

    Is AI Copyright Really Necessary? Is AI Copyright Really Necessary? Copyright laws and original author attribution sounds like a fair practice unless looked closely.

    Council Post: Shaping Tomorrow – The Transformative Potential Of Quantum Machine Learning By navigating these complexities, we can ensure that Quantum Machine Learning does not just become a part of our future but shapes it, driving us towards a more sustainable and technologically advanced society.

    Killed By Amazon (Part 1) Since the list is too exhaustive, we have made it a two-part series

    Our Mission Is To Bring About Better-Informed And More Conscious Decisions About Technology Through Authoritative, Influential, And Trustworthy Journalism. SHAPE THE FUTURE OF TECH CONTACT US ⟶

    AIM discovers new ideas and breakthroughs that create new relationships, new industries, and new ways of thinking. AIM is the crucial source of knowledge and concepts that make sense of a reality that is always changing.

    Our discussions shed light on how technology is transforming many facets of our life, from business to society to culture. Facebook Twitter Youtube Linkedin Instagram WHO WE ARE ABOUT US NEWSLETTERS PODCAST VIDEOS EVENTS CAREERS EDITORIAL CALENDAR CONTACT US OUR BRANDS MACHINEHACK BEST FIRM CERTIFICATION AIM RECRUITS AIM RESEARCH AIM LEADERS COUNCIL ASSOCIATION OF DATA SCIENTISTS PENETRATION AND MATURITY (PEMA) QUADRANT OUR SERVICES ADVERTISE WITH US BESPOKE EVENTS EMPLOYER BRANDING HACKATHONS RECRUITMENT SERVICES SYNDICATED RESEARCH ADVISORY SKILL ASSESSMENT OUR CONFERENCES CYPHER MACHINECON INDIA MACHINECON USA MACHINE LEARNING DEVELOPERS SUMMIT THE RISING | WOMEN IN TECH CONFERENCE DATA ENGINEERING SUMMIT AWARDS AI100 AWARDS DATA SCIENCE EXCELLENCE AWARDS 40 UNDER 40 DATA SCIENTISTS WOMEN IN TECH LEADERSHIP BEST FIRM FOR WOMEN IN TECH DATA ENGINEERING AWARDS FOR DEVELOPERS HACKATHONS PRACTICE SKILLS COURSES DISCUSSION BOARD JOB BOARD TECHNICAL BLOGS AIM CAMPUS AMBASSADOR © ANALYTICS INDIA MAGAZINE PVT LTD & AIM MEDIA HOUSE LLC 2023

    Terms of use Privacy Policy Copyright

  2. Assistant
    • Meta has released an open-source model called MMS: Massively Multilingual Speech for speech-to-text and text-to-speech in 1100 languages
    • MMS can recognize 4,000 spoken languages, reportedly with half the error rate of OpenAI's Whisper
    • Meta and its collaborators developed LIMA, a 65 billion parameter model fine-tuned with only 1,000 prompts/responses and without the need for reinforcement learning with human feedback (RLHF)
    • LIMA is claimed to be on par with GPT-4 and Bard in performance, but the paper comparing them is still under review
    • Meta has shifted its focus from developing proprietary chatbots to supporting the open-source community with LLaMa, an open-source LLM
    • Developers using LLaMa are creating models that reportedly outperform GPT-4 or Bard on less computation requirements

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.