Running Llama-7B on Windows CPU or GPU

This post is being written during a time of quick change, so chances are it’ll be out of date within a matter of days; for now, if you’re looking to run Llama 7B on Windows, here are some quick steps.

Code Repo:

Start by running PowerShell. Create a new directory and enter it.

mkdir llama
cd llama

I am assuming you have Python and PIP already installed, if not you can find steps on ChatGPT.

Next you need to create a Python virtual environment, you can do this without a virtual environment, but as of now it requires using nightly builds of Pytorch (for flash attention) and an unmerged branch of transformers.

python -m venv .venv

This should create and activate a virtual Python environment. Next we’re going to install everything you need:

pip install --pre torch torchvision torchaudio --index-url
pip install git+
pip install sentencepiece

This will take a few moments.

Now create a file called with the following body:

import transformers

device = "cpu"

tokenizer = transformers.LLaMATokenizer.from_pretrained("decapoda-research/llama-7b-hf")
model = transformers.LLaMAForCausalLM.from_pretrained("decapoda-research/llama-7b-hf").to(device)

batch = tokenizer(
    "The capital of Canada is",

batch = {k: for k, v in batch.items()}
generated = model.generate(batch["input_ids"], max_length=100)

That’s all there is to it! Use the command “python” to run it, you should be told the capital of Canada! You can modify the above code as you desire to get the most out of Llama!

You can replace “cpu” with “cuda” to use your GPU.

WordPress using Docker and Redis. Deploy the simple way.

I haven’t updated this site in a long time, but I’ve been paying a monthly hosting fee all along. This year, when the site came up for renewal with Bluehost, I felt their prices where high, and their pricing scheme was deceptive. I wasn’t able to switch packages to the lower tier on the dashboard, and when I called in the price quoted wasn’t the same price on the website. Before calling, I decided I’d not let the call run for more than 5 minutes, if they couldn’t figure it out that fast, I might as well self-host; I’ve been wanted to set this up a while anyway.

First, Get a Server

You’re going to need a server, I am going to assume you know how to get a server up and running. I used a nano AWS instance, but a basic Digital Ocean droplet would do nicely.

Prepare the Server

Now you’re going to need to do is install Docker, and Docker Compose. SSH in to the server and let’s get started. The lines below are meant for Amazon Linux, if you’re using something else, you will need to adjust accordingly:

# Update your packages, always good to start with this.
sudo yum update -y

# Install Docker
sudo yum install docker

# Start Docker
sudo service docker start 

# Allow the default ec2-user to interact with Docker
sudo setfacl --modify user:ec2-user:rw /var/run/docker.sock 

# Install Docker-Compose (all one line, wrapped because of WordPress)
sudo curl -L$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose

# Enable Docker-Compose to be executable
sudo chmod +x /usr/local/bin/docker-compose

# Make sure Docker-Compose is working
docker-compose version

# If you see "Docker Compose version v2.1.1", it worked!

Now you’ve got your machine all ready to go, let’s install the software

Optional Swap Space

If you’ve chosen a server that doesn’t have much memory, like me, you will probably see errors when you start the Docker network, I’ve added 2GB of swap space to fix this:

# Create a 2GB file
sudo fallocate -l 2G  swap

# Set the file permissions
sudo chmod 600 swap

# Turn it in to a swap
sudo mkswap swap

# Enable the swap
sudo swapon swap

Install WordPress + MySQL + Redis

Let’s install all the software.

# Create the directory for the files, chown it, and enter it
sudo mkdir /var/WordPress
sudo chown ec2-user:ec2-user /var/WordPress
cd /var/WordPress

# Open "docker-compose.yml" to editing
nano docker-compose.yml

The last line above should open a basic text editor called Nano. Copy and past the following contents in to the file, I will break down this file below. After pasting use “ctrl+x” , “y”, “enter” to save it.

version: "3.9"

    image: "wordpress:latest"
    restart: always
      - 80:80
      WORDPRESS_DB_USER: wp_user
      WORDPRESS_DB_PASSWORD: secret_password
      WORDPRESS_DB_NAME: wp_database
      - ./site:/var/www/html

    image: "mysql:5.7"
    restart: always
      MYSQL_DATABASE: wp_database
      MYSQL_USER: wp_user
      MYSQL_PASSWORD: secret_password
      - ./data:/var/lib/mysql

    image: "redis:alpine"

Version 3.9 is the version of the docker-compose, nothing special here. This creates a network of 3 containers; a WordPress container that maps port 80 of the host to the container. A “db” container, which consists of MySql 5.7, you might have noticed there is a random root password, you will not need to access the database as root, so we can do this. Lastly, a generic Redis container.

It’s not good practice to embed your secret password in the open like I did above. You should use Docker secrets for this, but for now, this will do.

Turn it on!

Use the following command to turn it on:

docker-compose up

You should see a lot of text on the screen, once it’s up and running, test it by visiting http://<SERVER IP> in your browser. You should see this:

The WordPress setup screen!

If this works, go back to your SSH session and hit ctrl+c to stop the network. Let’s run it in detached mode (so it stays running in the background):

docker-compose up -d

Now go back to your browser and go ahead and setup your WordPress instance, we still need to finish off setting up Redis. Continue once you’ve setup WordPress.

Enabling Redis for Max Performance

Go to “Plugins” and top the “Add New” button.

Search for “redis” and install “Redis Object Cache”

Don’t forget to activate it!

Now go to settings:

and enabled it:

We’re not yet done, this will result in a “Status: Not connected”. This is expected because by default this plugin tries to connect to localhost as the Redis server, we need to change it. Let’s go back to our SSH session.

# Edit to the wp-config.php file
sudo nano /var/WordPress/site/wp-config.php

# Add this after "<?php" to use "redis" as the hostname.
define('WP_REDIS_HOST', 'redis');

# ctrl+x , y , enter to save
Your file should look like this.

Finally, refresh the WordPress admin panel and you should have WordPress connected to Redis

Redis is connected.

That’s all! You can not setup your WordPress instance and can use it as you wish!

Building a Simple Soil Moisture Sensor

This article is a sub-article of another article (Growing Purple Strawberries).  In this article I build a simple soil moisture indicator.

Simple Moisture Monitor

The plan is to have a device that has a red LED when the soil of a plant is dry, and a green LED when there is adequate soil moisture.  This is a very simple circuit so I don’t need to make any sketches and can jump straight to a prototype:

As you can see, when I apply the moist tissue to the sensor the LED switches from red to green.  I used the following parts:

  • Arduino
  • RGB LED Light
  • YL-69 Moisture Sensor
  • YL-38 Signal Sensor/Decoder
  • 220Ω Resistor

And here is the Arduino source code:

void setup()
   pinMode(3, INPUT_PULLUP);
   pinMode(4, OUTPUT);
   pinMode(5, OUTPUT);

void loop()
   digitalWrite(4, !digitalRead(3));
   digitalWrite(5, digitalRead(3));

The YL-38 has an extremely useful calibration dial that can be used to set the level of moisture at which the digital bit is triggered.

Switching Over to a ATtiny85

At first I was going to build a permanent prototype using a Digispark, but the ATtiny85 had a lot of advantages over the Digispark and it will make a great microprocessor for this project.  I also bought a USBasp programmer for it that I haven’t yet used.  Benefits of to the ATtiny85 compared to the Digispark is that it is smaller, has a guaranteed 20KΩ input pull-up resistor on all pins, it’s cheaper, it doesn’t have an on-board LED that drains power, it is easy to set the clock speed to 1MHz to save power and it is very reliable.  The downside is that it can’t just be plugged in to the USB port for programming (but this also means I can build a fancy programming unit); it also doesn’t have an on-board testing LED light (not a big deal), finally, it doesn’t have an on-board 5V power regulator (this is a big one, I will need to figure out something for powering the ATTiny85).

I put together a USBasp programmer (and made a post on doing it too).

I can quickly program ATtiny chips by putting the chip in the IC socket and plugging an USBasp in the ribbon socket.
I can quickly program ATtiny chips by putting the chip in the IC socket and plugging an USBasp in the ribbon socket.

I changed the pin numbers and uploaded the program to the ATtiny in no time.  I used two AA batteries to power the project, this was more than enough power for the ATtiny85, but a little less for the soil sensor, which became very sensitive and needed an increased amount of moisture to trigger.  I will need to run some soil tests to determine if we can run this project of 3V or will need 5V.  Here it is running on an ATtiny:

And here is the new source code:

void setup()
  pinMode(0, INPUT_PULLUP);
  pinMode(1, OUTPUT);
  pinMode(2, OUTPUT);

void loop()
  digitalWrite(1, digitalRead(0));
  digitalWrite(2, !digitalRead(0));

The Power Struggle

I now need to find a way to power this whole thing, right now it is running off two AA batteries.  AA batteries have about 2,500 mAh of power; using a multimeter I can see the draw of this project is about 7.8 mA @ 3V.  Testing each component individually, it appears the ATtiny is comsuming about 1.1 mA, the LED about 2.5 mA and the rest by the sensor.

If we were to power this project as-is, using AA batteries it could theoretically run for 320 hours (it won’t because the voltage drops as the batteries age).  That isn’t good enough, we either need to reduce consumption significantly, or find a different power source; I believe we can reduce consumption by at least 95%.

We can get a huge savings by not running the LED and sensor continuously.  We can drop it’s duty cycle by a huge amount.  We can blink the LED for 500 ms every 10 seconds, meaning it will reduce power consumption of the sensor and the LED by 95% (We might even be able to get away with a smaller duty cycle).

After trying it out, it’s a win; realistically, the device would still work even if the duty cycle was 0.5ms per 30s.  Using a multimeter, I am seeing a current of about 1mA when the LED and sensor is off, and 7.7mA when it is on.  Meaning (based on a 30s cycle), 1.66% of the time we are using 7.7mA and 98.3% of the time, we are using 1mA.  This makes our average consumption 1.11mA; therefore, on 2x AA batteries we can run the device for 2,252 hours (or three months).

But wait, there’s more!  We can reduce power consumption even more.  At this point most of the power is being drained is by the continuous 1mA of the ATtiny85.  This draw can be reduce by putting the chip in low power mode, and putting it to sleep when the sensor and LED is off.

With a few updates to the code, I have the chip sleeping for 40s, using 0.004 mA and the system awake for 500ms consuming 5mA.  This means the average consumption of the system is just 0.33mA.  Now, if we put in two fresh AA batteries, they should last about 1 year; much more acceptable.

To save even more power I made a few final tweaks:

  1. Blink the LED instead of have it solid reduces consumption when the LED is on by 1/2 and it makes the LED more noticeable.
  2. Turn off the moisture sensor when the LED is blinking, the sensor only needs to be on for a few milliseconds to get the moisture reading.
  3. Reduce the sensor interval to 80 seconds, it is more than enough.

With all this done, I estimate my average current is 0.08mA, meaning on two batteries we will be able to run the device for about 3.5 years.

Now I just need to calibrate the sensitivity, solder it all together and we’re good to go! Here is the final code:

#include <avr/wdt.h>
#include <avr/sleep.h>

int cnt = 0;

void setup()
  pinMode(0, INPUT_PULLUP);  //The sensors goes low when the soil is moist, and must be pulled up otherwise
  pinMode(1, OUTPUT);        //The green LED (using a common anode RGB LED)
  pinMode(2, OUTPUT);        //The red LED (same RGB LED as the green one)

  pinMode(3, OUTPUT);        //We will use pin 3 as the negative for the sensor, so we can turn it off when not in use

  set_sleep_mode(SLEEP_MODE_PWR_DOWN);  //When we go to sleep, use SLEEP_MODE_PWR_DOWN, the mode that uses the LEAST amount of power
  sleep_enable(); //Allow the ATtiny to go to sleep.

  cli();  //Disable interupts temporarly why we do some important work
  wdt_reset();  //Reset the watchdog counter for good measure
  WDTCR  |= (1<<WDCE) | (1<<WDE); //Set these bits in order to set the watch dog configs
  WDTCR  = (1<<WDIE) | (1<<WDP3) | (1<<WDP0); //Set the watchdog to 8s.

void loop()
  if (cnt == 0) //Only 1 in 10 loops will actually check the soil, since 8s is the maximum watchdog interval and we want to run this code every 80s.  9 times out of 10, we just put the chip back to sleep.
    digitalWrite(3, LOW);//Turn on the sensor
    delay(10);  //Give it a moment, just to be safe
    bool moist = digitalRead(0); //Read the sensor
    delay(10);  //Give it a moment, just to be safe.

    digitalWrite(3, HIGH);     //Turn off sensor

    for (int i = 0; i < 20; i++) //Blink the LED 20 times
      //Set the LEDs based on the moisture reading
      digitalWrite(1, moist);
      digitalWrite(2, !moist);

      delay(50); //Wait 50ms

      //Turn off both LEDs
      digitalWrite(1, HIGH);
      digitalWrite(2, HIGH);

      delay(50);  //Wait 50ms

  ADCSRA &= ~(1<<ADEN); //Disable the ADC while in sleep (saves .25mA)
  sleep_mode(); //Enter sleep for 8s, reducing power consumption to 0.004mA!
  sleep_disable();  //Where to pick up after coming out of sleep.
  ADCSRA |= (1<<ADEN);  //Turn the ADC back on.

ISR(WDT_vect) // Watchdog timer interrupt.
  cnt++; //Increment a counter
  if (cnt == 10) { cnt = 0; } //Reset the counter every 10 interrupts

A  quick note on the code: I already programmed the chip and soldered it before realizing a couple of things. I tried to reprogram the chip with testing leads, but the wiring didn’t allow it.  The two things I noticed:

First, you don’t need so much green blinking, consider only 1 or 2 blinks for when the soil is moist (if that).  All the green blinking reminds me of Homer Simpson’s everything is okay alarm.

Second, you don’t need to turn on the ADC every time the watchdog wakes up the ATtiny.  You can probably get away with only turning it on on the cycles where the soil is tested; you may not even need it then (I wasn’t able to test this, but if anyone else does, let me know how it turns out).

Hardware Layout

I never made a circuit diagram for this device since it was so simple, but I will describe the circuit below:

  1. Two AA batteries to power the system.
  2. YL-38 Decoder:
    1. Plug in the YL-69 the only way it can be plugged in.
    2. Vcc goes to + of the battery.
    3. Ground goes to pin 3 of the ATtiny.
    4. Digital output goes to pin 0 of the ATtiny.
    5. Analog output is not connected to anything.
  3. RGB LED (Common anode):
    1. Vcc goes to + of the battery with the resistor between them.
    2. Red cathode goes to pin 1 of the ATtiny.
    3. Green cathode goes to pin 2 of the ATtiny.
  4. ATtiny:
    1. Vcc goes to + of the battery
    2. Ground goes to – of the battery

Test your circuit on a breadboard before soldering.

The Final Product

Here are a couple of pictures of the whole thing soldered together on the strawberry plant.

Soil Sensor Indicating Moist Soil
Soil Sensor Indicating Moist Soil

Soil Sensor Detailed Circuit
Soil Sensor Detailed Circuit