Use generative AI assistants like ChatGPT for FREE @ Cédric Walter | Wednesday, Feb 7, 2024 | 4 minutes read | 775 Words | Update at Wednesday, Feb 21, 2024

If you’re intrigued by the concept of AI assistants like ChatGPT, Google Bard, Bing Chat, or others, you might have certain apprehensions regarding privacy, expenses, and beyond. This is where Llama 2 steps in.

LLaMA (Language Model for Answering) is a powerful language model developed by Facebook that can understand and respond to natural language input in a conversational manner. It is the successor of the original LLaMA model, which was released in 2019.

LLaMA 2 builds upon the advancements made in the first version of the model, with several improvements in areas such as:

  1. Contextual Understanding: LLaMA 2 has improved contextual understanding abilities, enabling it to better comprehend the nuances of language and provide more accurate responses.
  2. Conversational Ability: The new model has been trained on a larger dataset and can engage in longer, more complex conversations than its predecessor.
  3. Emotional Intelligence: LLaMA 2 has been fine-tuned to recognize and respond to emotions expressed in text, such as empathy or sarcasm.
  4. Multi-Turn Responses: The model can now engage in multi-turn conversations, where it responds to multiple messages in a conversation without losing context.
  5. Improved Performance: LLaMA 2 has been optimized for better performance on a wide range of tasks, including question answering, text classification, and language translation.

Overall, LLaMA 2 represents a significant improvement over the original LLaMA model, with enhanced conversational abilities, emotional intelligence, and multi-turn response capabilities.

This model is trained on 2 trillion tokens, and by default supports a context length of 4096. Llama 2 Chat models are fine-tuned on over 1 million human annotations, and are made for chat.

Some use cases for software developers

  • Explain what this code does
  • Simplify it
  • Rewrite it in a more efficient way
  • Rewrite it in a more readable way
  • Replace the use of a library with another
  • Write documentation for it
  • Describe potential edge cases
  • Write unit tests for those edge cases

Memory requirements

  • 7b models generally require at least 8GB of RAM, 3.8GB download
  • 13b models generally require at least 16GB of RAM, 8GB download
  • 70b models generally require at least 64GB of RAM If you run into issues with higher quantization levels, try using the q4 model or shut down any other programs that are using a lot of memory.

How to run Llama 2 on your Mac, Linux or Windows for free using Ollama

If you have a Mac or Linux (Windows support coming), you can use Ollama to run LLaMA 2. It’s by far the easiest way to do it of all the platforms, as it requires minimal work to do so. All you need is a Mac and time to download the LLM, as it’s a large file.

Download Ollama

The first thing you’ll need to do is download Ollama . It runs on Mac and Linux and makes it easy to download and run multiple models, including Llama 2. Once Ollama is downloaded, move the ollama.app to the MacOS application folder and run it.

If you have 8GB RAM

We will then download a 7billion parameter LLaMA 2 model. Open a new Terminal and run (it will download a bit less than 4GB)

ollama run llama2

If you have 16GB RAM

We will then download a 13billion parameter LLaMA 2 model. Open a new Terminal and run (it will download a bit less than 8GB)

ollama pull llama2:13b

then run it with

ollama run llama2:13b

You’re4 done, you have a chat gpt like ai assisant in your terminal for free!

Running Ollama in a web browser

We will use docker compose

version: '3.8'

services:
  ollama:
    volumes:
      - ollama:/root/.ollama
    container_name: ollama
    pull_policy: always
    tty: true
    restart: unless-stopped
    image: ollama/ollama:latest

  ollama-webui:
    build:
      context: .
      args:
        OLLAMA_API_BASE_URL: '/ollama/api'
      dockerfile: Dockerfile
    image: ghcr.io/ollama-webui/ollama-webui:main
    container_name: ollama-webui
    volumes:
      - ollama-webui:/app/backend/data
    depends_on:
      - ollama
    ports:
      - ${OLLAMA_WEBUI_PORT-3003}:8080
    environment:
      - 'OLLAMA_API_BASE_URL=http://ollama:11434/api'
      - 'WEBUI_SECRET_KEY='
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

volumes:
  ollama: {}
  ollama-webui: {}

you can now access http://localhost:3003 and self register an admin account

Using text-generation-webui user interface for nearly any AI model

We will be using the project https://github.com/oobabooga/text-generation-webui

warning

Most serious and performing model need a beefy graphic card and lot of memory, dont expect to run them on a low or mid range GPU or older CPU

git clone https://github.com/oobabooga/text-generation-webui

then start either one of these scripts in a terminal

  • start_linux.sh
  • start_windows.bat
  • start_macos.sh
  • start_wsl.bat

now browse to http://127.0.0.1:7860

Go to the model tab, we need to load LLama 2 with 13Billion parameters first

go to Download model or LoRA and use TheBloke/Llama-2-13B-Chat-fp16 (32GB of download)

Click the download button and wait….

When finished, hit the reload button, select the model in the list box, and you’re done

Stable Difusion GUI

see https://github.com/AUTOMATIC1111/stable-diffusion-webui

Related content

© 1997 - 2024 Cédric Walter blog

Powered by Open Sources technologies

avatar

Cédric WalterA true selfless act always sparks another

6s a1 acma adaptability advocate-for-change ai airplane algorand alice-hlidkova-author alpine alps altruism-vs-commercialization antique-scooters antiseptic-rinse apache arcade arcade-gaming armattan art artemis artemis-viper artistic-expression atlassian authenticity-in-writing authenticity-matters bambulab bash bean bennu bernardet bestwishes betaflight betruger beware bio bioethics bitcoin blockchain blockchain-consensus-encyclopedia blockchain-systems blog book-review books bots Bought box brand-authenticity brand-integrity brand-protection breaking-barriers business-management business-milestones business-strategy business-success business-transformation businessbooks byzantine-fault-tolerance calculator calibre calibre-web camera case-studies cc2500 cgm-next challenges channel-setup cheaper cherry-blossoms choosing-fbl-gyro ci/cd classic-games classic-scooters classic-vespa climb climbing codefest collectible-scooters collectibles collector color competition consensus-algorithms consensus-mechanisms console consumer-awareness containerization contest control-surfaces controller copy counterfeit-awareness counterfeit-culture counterfeit-market counterfeit-vs-authentic covid19 creating cryptocurrency cultural-experience cultural-richness curve-adjustments customer-discovery cve-issues dance-dreams death decentralization decentralized dental-hygiene dependency Design development devfest devops distributed-ledger-technology diverse-perspectives diy-dental diy-health dji docker docker-compose docker-hosting docker-networking docker-registry docker-security dont-buy dotnet Download downloading dreams-and-reality drone dynamic-ip ecology edgetx elrs elta emotional-challenges emotional-hurdles empowering-narrative endpoints engelberg Ensitm entrepreneurial-lessons entrepreneurial-mindset entrepreneurs entrepreneurship entrepreneurship-books Essaim essentially ethereum ethical-dilemmas evoque execution exercise-form facebook failure-analysis failure-stigma failure-to-success fake fake-apparel fake-brands fake-goods family family-building family-dynamics fashion-ethics fashion-fraud fbl-controllers fbl-system-compatibility fbl-system-features fbl-system-reviews fertility-struggles finance-books financial-modeling financiallanning firearm firmware-customization firmware-issues fitness-routine fitness-tips flight-controller flybarless-advantages flybarless-systems foss fpv frame France freestyle fresh-breath friendship-goals front gallery game-music gameplay-mechanics gamer-community games gaming-culture gaming-enthusiast gaming-history gaming-legacy gaming-nostalgia generative-ai git global-impact google green-tea green-tea-mouthwash growth-hacking-books growth-mindset guide hackathon hackday hackfest health-and-wellness helicopter helicopter-community helicopter-gyro helicopter-tuning herbal-mouthwash hewlettpackard historical-scooters hobby hobbyist-blog holidays holistic-oralcare hollidays home-remedy home-workouts homelab homemade-oralcare honda honesty honey hornet how-to howTo https hugo human-connection hygiene-routine icecream iconic-scooters iflight iflightnazgulevoque immich indoor industrial-shit industry injury-prevention innovation innovation-books innovation-journey ios japan-travel japanese-cuisine jar java jdk11 jellyfin joint-health junit jupiter kitchen knee-rehabilitation knee-stability knockoff-alert kyoto lacoste lacoste-counterfeit lambretta landmarks leadership leadership-books lean-startup learning-from-failure leg-day leg-workouts legal-complexities legit-fashion let's-encrypt life-transformations link linux llm local-traditions m2evo macos magical-adventure magician-lord main make manurhin manurhin-sm75 mapping marathon market-research marketing-books maven me medical-advancements metakernel miami-entertainment mid-century-scooters migration mindset-shifts minimum-viable-product minty-fresh mixer-settings mk3 mk4 model-setup modern-family modern-motherhood moon moral-encounters motherhood-dilemmas motorcycle mount mountain mountains mouth-rinse mouthwash-ingredients mouthwash-recipe Mulhouse muscle-activation music mvs mycollection NASA natural-mouthwash nature nazgul neo-geo-aes neogeo network new-bookrelease nginx-proxy north-face north-face-replica nostalgic-scooters nv14 old-school-scooters omphobby open-source open-source-rc opensource opentx openvpn oral-care oral-health organizer osaka oss overcoming-challenges p1p p1s parental-rights parenthood-reflections parts patella-health persistence personal-relationships photos physical-therapy pivot-strategy pixel-art planet platform plex pluto pretty-girl-complex privacy product-market-fit productivity-books proof-of-stake proof-of-work protect-your-style prusa prusa-research public-image quadcopter quadriceps-strength radio-control radio-programming radiomaster rare-scooters raspberrypi raspbian rates-configuration rc rc-community rc-configuration rc-firmware RC helicopter rc-helicopter-electronics rc-helicopter-enthusiasts rc-helicopter-setup rc-helicopter-technology rc-helicopter-tips rc-helicopters rc-modeling rc-simulator realdebrid realflight receiver reflex-xtr refreshing-breath rehabilitation-exercises relationship-complexities released remote remote-control-flying reproductive-ethics resilience-in-business resilient-women restored-scooters retro-gaming retro-gaming-community retro-gaming-console retro-scooters reverse-proxy rhythms-of-life risk-management robotic router rx sab-raw-580 sab-raw-700 sales-books scooter-enthusiast scooter-memorabilia scooters security-nightmare self-leveling-helicopter server-configuration servo-config skydiving snk snk-corporation snk neo geo soap social-issues solex space spams sport ssl-termination ssl/tls startup-books startup-failure static-code-generator steam strategic-networking streaming strength-training success-stories sun support surrogacy-agency surrogacy-journey surrogacy-narratives swiftui swiss switzerland team team-building team-dynamics teeth-cleaning temples-and-shrines tendermint terrot tokyo traefik transmitter transmitter-firmware travel travel-tips tunnel turning-setbacks-into-success tutorial tx unconventional-strategies vacation velosolex vespa viaferrata video video-game-review vintage vintage-scooters vintage-two-wheelers vintage-vespa vintagegaming vmo-exercises warez web-security wind winner winterthur women-supporting-women wordpress workout-progression x1c zurich zyxel zyxel-avoid zyxel-not-serious-with-security zyxel-outdated zyxel-router-not-good
Me

Cédric Walter is a French-Swiss entrepreneur, investor, and software engineer based in Zurich, Switzerland. He spent his career developing software applications for Swiss insurance companies to handle billions of dollars in premiums. He cofounded Innoveo AG and as the software architect developed the no-code platform designed to reduce the manual coding that powers many software apps. As an active participant in the European hacking community, he works on many open source projects including blockchain. Cédric is a winner of multiple hackathons. His expertise include designing back end, event-based, and blockchain systems. Cédric is also the founded Disruptr GmbH, a software development company that offers full spectrum of services for businesses of all sizes.

JAVA full-stack developer since 2000, in Blockchain since 2017, Certified Scrum Master 2012, Corda Certified Developer in 2019, Ethereum smart contract expert in the SWISS Blockchain Security working group

Hackathons

  • HackZurich 2022 – Level Up in top 25 finalist among 134 submissions
  • SBHACK21 – SwiFi winner of best Solution on Algorand, overall Winner 3rd Prize, CV Labs Fast Track Ticket
  • HackZurich 2020 Europe’s Biggest Hackathon winner in category Migros
  • SBHACK19 – LendIt winner of Swiss biggest Blockchain Hackathon. On chain insurance and ledger for agricultural land soil.
  • Member of the Bitcoin Association Switzerland and Cryptovalley association Switzerland,

PGP: DF52 ADDA C81A 08A6

Copyright information

All editorial content and graphics on our sites are protected by U.S. copyright, international treaties, and other applicable copyright laws and may not be copied without the express permission of Cedric Walter, which reserves all rights. Reuse of any of Cedric Walter editorial content and graphics for any purpose without The author ’s permission is strictly prohibited.

DO NOT copy or adapt the HTML or other code that this site creates to generate pages. It also is covered by copyright.

Reproduction without explicit permission is prohibited. All Rights Reserved. All photos remain copyright © their rightful owners. No copyright infringement is intended.

Disclaimer: The editor(s) reserve the right to edit any comments that are found to be abusive, offensive, contain profanity, serves as spam, is largely self-promotional, or displaying attempts to harbour irrelevant text links for any purpose.

Others

If you like my work or find it helpful, please consider buying me a cup of coffee ☕️. It inspires me to create and maintain more projects in the future. 🦾

It is better to attach some information or leave a message so that I can record the donation 📝 , thank you very much 🙏.

Reproduction without explicit permission is prohibited. All Rights Reserved. All photos remain copyright © their rightful owners. No copyright infringement is intended.