NodeJS and fnm

Verdaccio is built using NodeJS, so we need to install node. I know I'll want to run other node based apps as well, and they might require a different version of node. So I'll install a node version manager first, and then use that to install node.

I'll be using Fast Node Manager fnm, which requires unzip to be available for its installation script to work

pistachio@verdaccio:~$ sudo apt install unzip

It's a pretty common practice on Linux to have specific users for specific apps/services. We'll be doing the same thing here, so let's create a separate user for Verdaccio.

pistachio@verdaccio:~$ sudo adduser --system --gecos "" --group verdaccio

This isn't a "normal" user that should be able to log in to the server, so we're creating a system user here. And we're skipping all the prompts for Full Name, Room Number etc by specifying --gecos "". Finally we're also creating a separate group for this user that we also name "verdaccio".

Let's switch over to the new user

pistachio@verdaccio:~$ sudo su -s /bin/bash verdaccio

Since verdaccio is a system user with no login switching is a little different in that we have to specify that we want to run the bash shell. Let's follow up by running cd to get to verdaccio's home directory.

Now we can install fnm by running its install script

verdaccio@verdaccio:~$ curl -fsSL https://fnm.vercel.app/install | bash

Notice that we're not using sudo here because we want to install this as the verdaccio user into directories that verdaccio (the user) has full access to.

Follow up by sourcing .bashrc to load fnm into our current shell (another option would be to log out and then log back in again)

verdaccio@verdaccio:~$ source /home/verdaccio/.bashrc

With fnm loaded we can now install node. I like using the latest LTS version, which is node 20 right now.

verdaccio@verdaccio:~$ fnm install 20

Verdaccio

Finally ready to install Verdaccio!

verdaccio@verdaccio:~$ npm install -g verdaccio

We install it globally, which just means that it's "global" to the verdaccio user, i.e. not tied to any particular nodejs project.

Make sure it's installed, and also print its installation directory

verdaccio@verdaccio:~$ npm ls -g --depth 0
/home/verdaccio/.local/share/fnm/node-versions/v20.12.2/installation/lib
├── corepack@0.25.2
├── npm@10.5.0
└── verdaccio@5.30.3

That's looking great! Let's run it 🙂

verdaccio@verdaccio:~$ verdaccio
 info --- config file  - /home/verdaccio/verdaccio/config.yaml
 info --- the "crypt" algorithm is deprecated consider switch to "bcrypt" in the configuration file. Read the documentation for additional details
 info --- using htpasswd file: /home/verdaccio/verdaccio/htpasswd
 info --- plugin successfully loaded: verdaccio-htpasswd
 info --- plugin successfully loaded: verdaccio-audit
 warn --- http address - http://localhost:4873/ - verdaccio/5.30.3

And now stop it again by pressing Ctrl+C. When we just now ran Verdaccio for the first time it created the directory /home/verdaccio/verdaccio. It contains Verdaccio's config file and a directory it'll use for storing packages it downloads or someone publishes to it. For now we want to focus on the config file. There are a few settings we want to tweak.

verdaccio@verdaccio:~$ vim verdaccio/config.yaml

As you might have seen when you ran Verdaccio it said something about switching to bcrypt. Do that by finding the commented out # algorithm: bcrypt line and removing the comment (#) from the start of that line. Also uncomment the rounds setting a couple of lines down. This is what you want that section of the config file to look like

# https://verdaccio.org/docs/configuration#authentication
auth:
  htpasswd:
    file: ./htpasswd
    # Maximum amount of users allowed to register, defaults to "+inf".
    # You can set this to -1 to disable registration.
    # max_users: 1000
    # Hash algorithm, possible options are: "bcrypt", "md5", "sha1", "crypt".
    algorithm: bcrypt # by default is crypt, but is recommended use bcrypt for new installations
    # Rounds number for "bcrypt", will be ignored for other algorithms.
    rounds: 10

Next up is logging. By default it just logs to stdout, but for our purposes it's better if it logs to a file

# https://verdaccio.org/docs/logger
# log settings
# log: { type: stdout, format: pretty, level: http }
log: { type: file, path: verdaccio/verdaccio.log, level: info }

Because we're running Verdaccio behind a reverse proxy we should configure trustProxy. If we don't configure it you won't see proper IP addresses in the Verdaccio logs, you'll only see the IP address the reverse proxy is forwarding all requests to (that was ::1 in my case).

server:
  keepAliveTimeout: 60
  # Allow `req.ip` to resolve properly when Verdaccio is behind a proxy or load-balancer
  # See: https://expressjs.com/en/guide/behind-proxies.html
  trustProxy: true

We're going to publish a test package later. To avoid trying to also publish to the upstream npmjs.org registry we're disabling proxying for packages in our own scope (I chose @tobbe.dev as my scope name, you're of course free to choose whatever you like)

packages:
  '@tobbe.dev/*':
    # our own scoped packages
    access: $all
    publish: $authenticated
    unpublish: $authenticated

(Without this config all scoped packages would use the '@*/*' rules, which includes proxy: npmjs which would also proxy any publish requests to the main npmjs.org registry.)

nginx

Now that Verdaccio is installed it's time to go back to configuring nginx again. nginx keeps all the config for the different (sub-)domains it serves in /etc/nginx/sites-available/ so let's go there (exit out of the verdaccio user shell if you're still in there first, so you get back to pistachio, exit)

pistachio@verdaccio:~$ cd /etc/nginx/sites-available/

Make a copy of default and call it pistachio.tobbe.dev for the subdomain we'll be using for Verdaccio. And then open it up in your editor of choice (vim, of course! 😄)

pistachio@verdaccio:/etc/nginx/sites-available$ sudo cp default pistachio.tobbe.dev
pistachio@verdaccio:/etc/nginx/sites-available$ sudo vim pistachio.tobbe.dev

Get rid of most of the comments and make sure everything is nicely formated so it's easy to read. This is what you want it to look like.

server {
    server_name pistachio.tobbe.dev; # managed by Certbot

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot

    location / {
        proxy_pass http://localhost:4873/;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $host;
        proxy_set_header X-NginX-Proxy true;
    }

    ssl_certificate /etc/letsencrypt/live/pistachio.tobbe.dev/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/pistachio.tobbe.dev/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    server_name pistachio.tobbe.dev;

    listen 80;
    listen [::]:80;

    if ($host = pistachio.tobbe.dev) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    return 404; # managed by Certbot
}

To activate this configuration we put a symlink to it in /etc/nginx/sites-enabled

pistachio@verdaccio:/etc/nginx/sites-available$ sudo ln -s /etc/nginx/sites-available/pistachio.tobbe.dev ../sites-enabled/pistachio.tobbe.dev

Now that we've moved the verdaccio server config to its own file, we should clean up the default config

pistachio@verdaccio:/etc/nginx/sites-available$ sudo vim default

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;
    return 410;
}

That config will make nginx reply with HTTP status 410 (GONE) for all sub domains that are not configured

Verify that your config is sound

pistachio@verdaccio:~$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Reload the nginx unit

pistachio@verdaccio:~$ sudo systemctl reload nginx

Time to log back into the verdaccio user account

pistachio@verdaccio:~$ sudo su -s /bin/bash verdaccio
verdaccio@verdaccio:/home/pistachio$ cd

Run Verdaccio temporarily in the foreground, just to make sure everything is working

verdaccio@verdaccio:~$ verdaccio

Open another terminal, SSH to your vps and look at the verdaccio logs to see that it started, and to be able to troubleshoot any problems

pistachio@verdaccio:~$ sudo tail -f /home/verdaccio/verdaccio/verdaccio.log
[sudo] password for pistachio:
 info --- config file  - /home/verdaccio/verdaccio/config.yaml
 info --- using htpasswd file: /home/verdaccio/verdaccio/htpasswd
 info --- plugin successfully loaded: verdaccio-htpasswd
 info --- plugin successfully loaded: verdaccio-audit
 warn --- http address - http://127.0.0.1:4873/ - verdaccio/5.30.3

Now try to access Verdaccio in your browser by going to your configured (sub-)domain. For me that's https://pistachio.tobbe.dev

You should see something like this:

Go ahead and follow the instructions:
On your local computer, in your terminal, run the following command (but substituting your own URL, of course)

❯ npm adduser --registry https://pistachio.tobbe.dev

Going back to the terminal where you're tailing your verdaccio logs you should now see

 info <-- 217.201.48.96 requested 'POST /-/v1/login'
 info <-- 217.201.48.96 requested 'PUT /-/user/org.couchdb.user:tobbe'
 info --- the user tobbe has been added

Now let's try publishing a package!

On your local computer

❯ mkdir -p ~/tmp/v-test
❯ cd ~/tmp/v-test
❯ npm init -y

Add a bin to package.json and update some values. The most important value to update is the name, to make sure it also includes our scope. Otherwise publishing will fail.

❯ vim package.json

{
  "name": "@tobbe.dev/v-test",
  "version": "1.0.0",
  "description": "Saying Hello World! from a Verdaccio package",
  "main": "index.js",
  "bin": {
    "v-test": "index.js"
  },
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "Tobbe Lundberg",
  "license": "ISC"
}

Add an index.js file

❯ vim index.js

#!/usr/bin/env node

console.log("Hello World!");

Now we're ready to publish our test package

❯ npm publish --registry https://pistachio.tobbe.dev/

Hopefully that goes well. If not – check your logs (both nginx and verdaccio) – to try to figure out what went wrong

Let's make sure it's in Verdaccio's storage. tree is a very useful Unix tool to visualize directory structures. So I recommend installing that first. This is what it looks like for me after a successful publish.

pistachio@verdaccio:~$ sudo apt install tree
pistachio@verdaccio:~$ sudo tree /home/verdaccio/verdaccio/storage
/home/verdaccio/verdaccio/storage
└── @tobbe.dev
    └── v-test
        ├── package.json
        └── v-test-1.0.0.tgz

2 directories, 2 files

Now that we know it's all working there's another tool we should configure – systemd – to make sure Verdaccio starts on server start and restarts if it ever crashes.

systemd

Verdaccio comes with a basic systemd unit file. We just need to copy it to a place where systemd will read it

pistachio@verdaccio:~$ sudo cp /home/verdaccio/.local/share/fnm/node-versions/v20.12.2/installation/lib/node_modules/verdaccio/systemd/verdaccio.service /etc/systemd/system/

You might be asking how I figured out that long path. And that's a good question! 🙂 You can run which verdaccio as the verdaccio user and you'll get something like /home/verdaccio/.local/state/fnm_multishells/27702_1713942027155/bin/verdaccio back. But because of how fnm works that's just a symlink and it's not guaranteed to always be available. To figure out where it really is we can combine that with realpath. So the final command is realpath `which verdaccio` and that'll give you /home/verdaccio/.local/share/fnm/node-versions/v20.12.2/installation/lib/node_modules/verdaccio/bin/verdaccio. So that's the full path to the bin. Removing the last two parts of that path gives us the base directory for Verdaccio. And then we can just add on systemd/verdaccio.service to get the full path to the systemd unit file 😅

Now I like to make some modifications to the default unit

pistachio@verdaccio:~$ sudo vim /etc/systemd/system/verdaccio.service

[Unit]
Description=Verdaccio lightweight npm proxy registry
After=network-online.target

[Service]
Type=simple
Restart=on-failure
RestartSec=5
User=verdaccio
WorkingDirectory=/home/verdaccio
ExecStart=/home/verdaccio/.local/share/fnm/node-versions/v20.12.2/installation/bin/node /home/verdaccio/.local/share/fnm/node-versions/v20.12.2/installation/lib/node_modules/verdaccio/bin/verdaccio

[Install]
WantedBy=multi-user.target

I added the After=... option, because we need network to be online before we can start Verdaccio. I also added RestartSec=5 to give the system a few seconds to recover if something goes wrong (like if something crashed). WorkingDirectory=/home/verdaccio is also new, and finally I changed ExecStart to run node and pass it the main entry point for Verdaccio. To figure out the node path I used realpath `which node` (just as I showed for verdaccio earlier in this guide). Just remember to run it as the verdaccio user.

To make sure the systemd unit is working make sure you don't have Verdaccio running anywhere. Close or at least log out of all terminals that are connected to your VPS except for one, where you're logged in as the normal user (pistachio in my case). Then reload systemd to read all new units and finally enable the Verdaccio one

pistachio@verdaccio:~$ sudo systemctl daemon-reload
pistachio@verdaccio:~$ sudo systemctl enable --now verdaccio

After running that you should once again be able to access the Verdaccio web interface at your configured (sub-)domain. And now you should see your published package there!

Final test

To really make sure everything is working properly we're going to do two things:

1. Reboot the server

2. Make sure you can npm install the newly created package without manually touching anything on the server

Let's get to it!

pistachio@verdaccio:~$ sudo reboot

Wait a couple of minutes while the server reboots, and then try accessing the Verdaccio web interface. When that's up you will be able to also install the package you published.

❯ npm i -g --registry https://pistachio.tobbe.dev @tobbe.dev/v-test

added 1 package in 451ms

And now try to run it:

❯ v-test
Hello World!

It works! 🎉 Now go celebrate 🎉 🥳 🪅

When you're back it's time to clean up (the boring work you have to do after any party 🙁) because you probably don't want that test package to be cluttering up your system, right? At least I didn't!

❯ cd ~/tmp/v-test
❯ npm uninstall -g @tobbe.dev/v-test
❯ npm unpublish --force --registry https://pistachio.tobbe.dev
npm WARN using --force Recommended protections disabled.
- @tobbe.dev/v-test@1.0.0
❯ cd
❯ rm -fr ~/tmp/v-test

Conclusion

Lot's of things going on in this part of the guide! We started with installing Node by using fnm. Then we installed and configured Verdaccio. After that we went back to nginx and did some major changes to its config to support our own (sub-)domain. To make Verdaccio more resilient to crashes and have it running automatically after server reboots we added a systemd unit for it. Finally we tested it all out by publishing a package, rebooting the server and then installing the package we had published.

You now have Verdaccio configured and running on a Hetzner Cloud server. Congratulations! What you want to do from here on is up to you 🙂 In the next (and final, for now at least) part of this guide I'll show you some things I needed to do for what I wanted to use this for.

Login and update

If you've followed along with Part 1 and Part 2 your server should be secure. Let's make sure it's also up to date!

pistachio@verdaccio:~$ sudo apt-get update && sudo apt-get upgrade

Answer Yes if/when it asks if you want to continue

If it asks about your sshd_config file, you want to keep your locally modified version. Press Tab to highlight <Ok> and then press Enter to continue

Following that it might ask about daemons using outdated libraries. Again, press Tab to highlight <Ok> and then press Enter to continue.

nginx installation

pistachio@verdaccio:~$ sudo apt install nginx

Again, it might ask you about daemons using outdated libraries. Select all of them and then continue

Remember that firewall we configured before? It's blocking access to nginx, so we need to update its config.

pistachio@verdaccio:~$ sudo ufw app list
Available applications:
  Nginx Full
  Nginx HTTP
  Nginx HTTPS
  OpenSSH

You might be tempted to go with HTTPS only, but we need HTTP to be able to verify our SSL setup later. Plus we will tell nginx to redirect all HTTP traffic to HTTPS, so we're going to allow "Nginx Full".

pistachio@verdaccio:~$ sudo ufw allow 'Nginx Full'
Rule added
Rule added (v6)

Check the firewall status

pistachio@verdaccio:~$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
Nginx Full                 ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)
Nginx Full (v6)            ALLOW       Anywhere (v6)

You should now be able to access your web server in your browser by going to http://<IP-address>

Depending on your domain/DNS you might also be able to access it using http://your.domain.tld but since I'm using a .dev top level domain (TLD) I can't, because they require SSL

SSL

Certbot will manage our SSL certificates.

pistachio@verdaccio:~$ sudo apt install certbot python3-certbot-nginx

pistachio@verdaccio:~$ sudo certbot --nginx -d pistachio.tobbe.dev

And this is the output you want

Let's take a look at the default nginx config

pistachio@verdaccio:~$ sudo less /etc/nginx/sites-available/default

You'll see a bunch of lines with # managed by Certbot at the end. Those were added thanks to the --nginx command line option we used when we ran the certbot command. The file is a bit messy with commented config, indentation a bit all over the place etc. But we'll create our own config soon, and make sure to clean it up so it's easier to follow along with what's going on.

For now we should just make sure it all works by going to https://pistachio.tobbe.dev in our web browser.

Conclusion

This was a pretty short part. We installed nginx and configured it to use SSL. We also verified that we could now access our web server using https. We'll come back to the nginx config, but first we need to install Verdaccio, and that's exactly what we'll do in the next part of this guide.

Cover Photo by Taylor Vick on Unsplash

Adding a User

As a rule of thumb you want to use the root user for as few things as possible, so let's go ahead and create a regular user account we can use instead. I'll name mine "pistachio". Make sure you give the user a strong password. You can skip all the other fields by just pressing Enter on all of them.

root@verdaccio:~# adduser pistachio
Adding user `pistachio' ...
Adding new group `pistachio' (1000) ...
Adding new user `pistachio' (1000) with group `pistachio' ...
Creating home directory `/home/pistachio' ...
Copying files from `/etc/skel' ...
New password:
Retype new password:
passwd: password updated successfully
Changing the user information for pistachio
Enter the new value, or press ENTER for the default
    Full Name []:
    Room Number []:
    Work Phone []:
    Home Phone []:
    Other []:
Is the information correct? [Y/n]
root@verdaccio:~#

Next thing we'll do is add the user to the sudo group

root@verdaccio:~# usermod -aG sudo pistachio

You can test sudo access by switching over to the new user, try to run a command that requires root privileges and then running again with sudo.

root@verdaccio:~# su - pistachio

pistachio@verdaccio:~$ ls -la /root
ls: cannot open directory '/root': Permission denied

pistachio@verdaccio:~$ sudo ls -la /root
[sudo] password for pistachio:

Enter the password you selected for your user. If everything works correctly the contents of the /root directory should now be printed to the screen, like .bashrc, .profile, etc

Something like this is what you should see

Run exit to go back to your root shell. One more thing we want to set up is ssh key login for the new user. We already have the info we need on the root user, so we can just copy that over and then make sure all owners and permissions are correct

root@verdaccio:~# cp -r ~/.ssh /home/pistachio
root@verdaccio:~# chown -R pistachio:pistachio /home/pistachio/.ssh

When that's done you can run exit again to log out of your ssh session.

Now let's try to ssh in as your new user ssh -i <path to private key> <username>@<IP-address>. With my example values that becomes

❯ ssh -i ~/.ssh/id_ed25519-hetzner-verdaccio pistachio@78.47.20.27

That should log you in with your ssh key, so without requiring the pistachio user password.

Firewall

Once you're in we're going to continue making our server more secure. Next step is to enable the Uncomplicated Firewall ufw.

pistachio@verdaccio:~$ sudo ufw allow OpenSSH
[sudo] password for pistachio:
Rules updated
Rules updated (v6)

pistachio@verdaccio:~$ sudo ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup

pistachio@verdaccio:~$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)

All connections, except SSH, are now blocked. When you install new applications you want to be able to connect to you need to activate them in the firewall.

SSH Server (sshd) Configuration

To enhance the security of your server even more, we'll deactivate password based logins, and not allow the root user to login using ssh at all

Open the sshd config file using your preferred editor. I use vim

pistachio@verdaccio:~$ sudo vim /etc/ssh/sshd_config

Find the commented out line that says
#PasswordAuthentication yes
and change it to
PasswordAuthentication no

Also find
#PermitRootLogin prohibit-password
and change that to
PerminRootLogin no

:wq to write the file and quit vim

Restart the sshd service

pistachio@verdaccio:~$ sudo systemctl restart ssh

Conclusion

Any computer connected to the Internet is subject to attacks. Common attack vectors are poorly configured SSH servers and (default) users with weak passwords. In this part of the guide we locked down the server as much as we could by enabling the Uncomplicated Firewall (ufw), making sure no one can login in remotely using only a username plus password combination and we blocked root login even over SSH.

Next up, in Part 3, is to configure a reverse proxy with SSL certificates to have encrypted communication with the Verdaccio NPM registry. We'll be using the nginx reverse proxy and web server for this.

Cover Photo by Sammyayot254 on Unsplash

Start by going to https://www.hetzner.com/cloud/ and signing up

Click the red "Sign Up" button, or "Login" in the top navigation bar and then "Cloud". Both buttons should take you to the screen below where you need to click "Register Now" and then fill in your email etc. It'll also ask for a credit card number, but you won't be charged yet.

Once you're through the registration process you should see something like this

Click "Create Server" and chose your desired location. Then pick Ubuntu 22.04 as your desired OS Image. For "Type" the cheapest option is absolutely good enough. So I went with a Shared Intel vCPU with 2 GB or RAM, 20 GB of disk space (SSD) and 20 TB of traffic (I wish I had that much traffic!).

To simplify things, and make sure my registry can be used by as many as possible, I also chose to pay for an IPv4 address. And I suggest you do the same.

Now it's time to add an SSH key. You don't have to, but as they say – it is recommended for security reasons. If you already have one you can go ahead and add it, otherwise you can generate a new one by pasting the text below into your terminal.

❯ ssh-keygen -t ed25519 -C "hetzner-verdaccio"

The output will look something like this

As you can see I also gave my key a custom name that matches the comment (-C) I passed to ssh-keygen, just to make it easier to identify. And you probably should use a passphrase, but for convenience I didn't this time 😬

Copy your public key. If you're on a MacOS computer you can do it like this:

❯ cat ~/.ssh/id_ed25519-hetzner-verdaccio.pub | pbcopy

Click "Add SSH key" on the server creation page and then paste your key into the form.

It'll parse the key for you and populate the Name field too. Choose "Set as default key" if you want and then click "Add SSH key". You should now see this

That's almost all we have to do. Now just scroll to the bottom and give your server a more descriptive name, like "verdaccio"

Finally click "Create & Buy now"

After a few seconds you should see something like this

Copy your server's IP address by clicking on it. You'll need it to connect to your server from the terminal.

So open up a new terminal and enter ssh -i ~/.ssh/id_ed25519-hetzner-verdaccio root@<IP-address> where you replace <IP-address> with the address you copied from the Hetzner web interface. It'll ask you to confirm the fingerprint. Type yes and press Enter.

You're now logged into your very own Hetzner Cloud VPS!

Conclusion

You've now bought a cloud computer on Hetzner. It's a very cheap Virtual Private Server (VPS) more than capable of hosting your own NPM registry and a few other things if you want. During setup we added an SSH key, so that you can login to the server and get root access without relying on potentially insecure passwords.

Next up in Part 2 we're going to create a regular user (not root) and make the server more secure.

Cover Photo by Matthieu Beaumont on Unsplash

Realtime/WebSocket support in Redwood has been requested since forever*. For most people, using something like https://pusher.com/channels or https://supabase.com/docs/guides/realtime is the best choice. But I wanted to see if I could roll my own. (*beginning of 2020 😄)

I decided to build a simple multiplayer card game that would use WebSockets to sync the game state for all players (what cards they have on hand, what cards they've played etc). That's what you can see in the screenshot at the top of the page.

EDIT 2023-02-27: A lot of the setup shown in this blog post can now be done for you by just running yarn dlx rw-setup-ws. Read more here https://community.redwoodjs.com/t/experiment-websockets-with-redwoodjs/4648

This blog post will show you how I set it up. But to keep it simple I'll just focus on a tiny part of the game: the score. If you look at the screenshot above you can see a score next to each player's name. I kept it super basic by just syncing all of the text inputs. You can just write whatever number you want in any of them and it'll sync to all players. We'll build something similar. An input field for a player's name and another one for the player's score. When the score is updated it'll be synced to all other players. But in this version, unlike in my actual game, you can only change your own score. (Not strictly true, but I'll let you figure that out as you play around with the code 😉)

Create a new Redwood project yarn create redwood-app --ts --git rw-ws-score

Redwood uses Fastify for its API side web server. To get it to understand WebSockets we need to install a plugin: yarn workspace api add @fastify/websocket

That's all we have to do as far as setup goes. Now let's write some code! This is where it really shows that Redwood wasn't designed for realtime or WebSocket support – we'll be writing all of the ws code on the api side in what is basically just meant to be a configuration file. But if you want ws support let us (the RW core team) know, and if there's enough demand we'll try to make the experience better!

Open up api/server.config.js and register the ws plugin.

const configureFastify = async (fastify, options) => {
  if (options.side === 'api') {
    fastify.log.info({ custom: { options } }, 'Configuring api side')

    fastify.register(require('@fastify/websocket'))
  }

  // ...
}

This lets Fastify know about WebSockets. Now we need to add a route that'll handle the ws traffic.

const configureFastify = async (fastify, options) => {
  if (options.side === 'api') {
    fastify.log.info({ custom: { options } }, 'Configuring api side')

    fastify.register(require('@fastify/websocket'))

    fastify.register((fastify) => {
      fastify.get('/ws', { websocket: true }, (connection) => {
        connection.socket.on('message', (message) => {
          console.log(`/ws message: ${message}`)
        })

        connection.socket.on('close', () => {
          console.log('Client disconnected')
        })
      })
    })
  }

  // ...
}

Let's test our code! Run yarn rw dev api in a terminal to start just the api side. In another terminal you can run npx -y wscat -c ws://localhost:8911/ws. It should open a connection to your server and give you a prompt. Type something, like hello world, and you should see it printed in the terminal that's running the RW api server.

Do you see the messages you type echoed in the api server terminal? If you do: Congratulations! You have a WebSocket web server running! If you don't: Try adding some console.logs, then restart the api server and try again. If you still can't get it to work, create a post on the RW forums and ping me there. We'll all do our best to help you.

Now that you've confirmed that the api side is working, let's add some code to the web side too. I'll over-engineer this a little bit for our simple example, just to give you a better foundation to stand on when you build this out with more functionality.

I'll put the websocket code in a React Context so that you can access it from anywhere in your app.

// web/components/WsContext/WsContext.tsx

import { useCallback, useContext, useEffect, useRef, useState } from 'react'

interface WsContextProps {
  players: Record<string, string>
  setScore: (playerId: string, score: string) => void
}

const WsContext = React.createContext<WsContextProps | undefined>(undefined)

interface Props {
  children: React.ReactNode
}

const WsContextProvider: React.FC<Props> = ({ children }) => {
  const [players, setPlayers] = useState<Record<string, string>>({})

  const ws = useRef<WebSocket>()

  useEffect(() => {
    const socket = new WebSocket('ws://localhost:8911/ws')

    socket.onopen = (event) => {
      console.log('socket open', event)
    }
    socket.onclose = (event) => {
      console.log('socket close', event)
    }
    socket.onerror = (event) => {
      console.log('socket error', event)
    }
    socket.onmessage = (event) => {
      console.log('socket message', event.data)

      try {
        const players = JSON.parse(event.data)
        setPlayers(players)
      } catch (e) {
        console.error('JSON.parse error', e)
      }
    }

    ws.current = socket

    return () => {
      socket.close()
    }
  }, [])

  const setScore = useCallback((playerId: string, score: string) => {
    ws.current?.send(JSON.stringify({ playerId, score }))
  }, [])

  return (
    <WsContext.Provider value={{ players, setScore }}>
      {children}
    </WsContext.Provider>
  )
}

export function useWsContext() {
  const context = useContext(WsContext)

  if (!context) {
    throw new Error(
      'useWsContext must be used within a WsContextProvider'
    )
  }

  return context
}

export default WsContextProvider

Basically what's going on here is I create a React Context with a useEffect that will only be called after the very first render thanks to its empty dependency array. The useEffect function sets up a new WebSocket connection to our api server, registers a bunch of callbacks and returns a clean-up function that will close the web socket connection when the context unmounts. Notice that we use the ws protocol when we specify the server address. The main socket callback is socket.onmessage which will receive all the messages from the WebSocket server. The other callbacks are mainly there to show you what's available and to help you debug any errors you might run into. On the context you can access all players and their scores plus a method to update the score for a player. It might look weird to have the score be a string instead of number, but that's just to make it easier to handle the score input later on. It makes it easier to handle empty input fields. WebSockets can only communicate using text, that's why we use JSON.stringify before sending an updated score for a player. Finally, there's a custom React hook to make it more convenient to use the context.

To make this context available to the entire app I place it in App.tsx

const App = () => (
  <FatalErrorBoundary page={FatalErrorPage}>
    <RedwoodProvider titleTemplate="%PageTitle | %AppTitle">
      <RedwoodApolloProvider>
        <WsContextProvider>
          <Routes />
        </WsContextProvider>
      </RedwoodApolloProvider>
    </RedwoodProvider>
  </FatalErrorBoundary>
)

The final piece of the web-side puzzle is to actually use the context. Let's create a page for this.

yarn rw g page home /

Here's the base component code

import { useState } from 'react'

import { useWsContext } from 'src/components/WsContext/WsContext'

const HomePage = () => {
  const [name, setName] = useState('')
  const [score, setScore] = useState('')
  const ws = useWsContext()

  return (
    <>
      <label>
        Name:{' '}
        <input
          value={name}
          onChange={(event) => {
            setName(event.target.value)
          }}
        />
      </label>
      <br />
      <br />
      <label>
        Score:{' '}
        <input
          value={score}
          onChange={(event) => {
            const score = event.target.value

            // Set the score in component state to make the input
            // value update immediately
            setScore(score)

            // Send to the server to update all clients (including
            // this one)
            ws.setScore(name, score)
          }}
        />
      </label>
      <hr />
      <ul>
        {Object.entries(ws.players).map(([playerId, playerScore]) => (
          <li key={playerId}>
            {playerId}: {playerScore}
          </li>
        ))}
      </ul>
    </>
  )
}

export default HomePage

Unless you're super new to React the code should be pretty straightforward. A couple of input fields with state bound to them and then the context we created earlier is used to get a list of players to map over to display their names/ids and scores. The one thing that might look a little unusual is how we both keep our own score in the local score state as well as send it to the web socket using ws.setScore. But I hope the code comments are clear enough in describing what's going on and why I do it like that. With this code in place, we can start the RW dev server again, but this time we start both sides yarn rw dev. Enter your name and a score and you should see something like /ws message: {"playerId":"Tobbe","score":"5"} in your dev server terminal output.

Now it's time to focus on the API side code again. It needs to keep track of all players and their scores and broadcast it to everyone. So I add two global variables to the Fastify server config file: players for all the players and their scores, and wsConnections to keep track of all connections I should broadcast the new scores on. Both those variables are used in the socket.on('message' handler function. The code for that function now looks like this

console.log(`/ws message: ${message}`)

try {
  const player = JSON.parse(message)
  wsConnections[player.playerId] = connection
  players[player.playerId] = player.score

  Object.values(wsConnections).forEach((wsConnection) => {
    wsConnection.socket.send(JSON.stringify(players))
  })
} catch (e) {
  console.error('Could not parse input', message)
  console.error('error object:', e)
}

Start the dev server again if it isn't still running and go to http://localhost:8910 in two different browser windows. Enter a name and a score in both of them and notice that the score immediately is synced between both windows 🎉

There are a few things I've left as an exercise for the reader 😁

• If you update your name you're seen as a new player and your old name with your old score is still kept around. See if you can make that work a little better

• You're allowed to enter text in the score input fields. Should probably only be allowed to enter actual numbers (except for the empty string (''), which you might still want to allow)

• Maybe you don't want to include yourself in the list of players with their scores. So maybe filter out your own player id.

• Make it so you can't "steal" someone else's name.

This should work pretty well running on your computer using the dev server. If you want to deploy it there are a few more hoops you have to jump through. I'll write another blog post about that.

The full code for the example can be found here https://github.com/Tobbe/rw-ws-score/

Thanks for reading and don't hesitate to reach out if you have any questions.

Let's say you have an e-commerce storefront with URLs like /products/redwoodjs-lapel-pin, /products/redwoodjs-skater-hat and products/redwoodjs-logo-shirt.

The RedwoodJS Shop is not actually built with the technique I'm demonstrating here, I just used it as an example of what one could do

You have probably defined the route that uses those URLs like this: <Route path="/products/{sku}" page={ProductPage} name="product" prerender />. The page probably passes the sku from the URL on to a <ProductCell sku={sku} /> to fetch details about the product from your database. So you'd need to be able to tell Redwood what sku(s) to pre-render for, and then Redwood has to know how to execute the GraphQL query in the Cell, with the given sku, connect to your database, return the data, and finally render it all on a page. If you use the latest canary version of Redwood all this is now possible!

As you know, Redwood uses React for its frontend. And the way pre-rendering works with React (and most other JavaScript frontend frameworks) is you generate a static html page and send that to the user's browser. This html page has a <script> tag (or several) that downloads the React runtime plus all the javascript logic that you have written for this particular page. It then replaces the static html on the page with dynamic html rendered by the React runtime. When this is done the page is finally ready for user interaction. And now, when you click on a link on the page to navigate somewhere else, this new page you're navigating to will be fully rendered by React. So no matter if you go to another page that is also pre-rendered or not, the existing html that's stored on the server will never be sent to the end user. This is great, because the page stays interactive, and only parts of the page that actually changes when you navigate (i.e. not your side menu, top navigation bar, footer etc) will update. The user gets a smoother experience, and you don't need such a beefy server as you offload a lot of the rendering to the user's client instead.

But what if your page is pretty much all static, and overall very basic? You don't need the interactivity that React brings. And it wouldn't be any rendering work that needed to be done on the server - it would just have to serve static html. In this case you might wonder - why use React at all? And that's a valid question. But let's ignore that for now, or I wouldn't have anything to blog about 😛 (In reality the argument is probably that you have some pages that are very basic, and/or have to be super fast, and you have other pages that are highly dynamic, and you want to use the same tech stack for all of them.) In this case it might make sense to just render pure html pages and statically serve them straight from your web-server like it was 1995 allover again.

Image from https://thehistoryoftheweb.com/a-mini-browser-for-the-masses/

Redwood's current pre-render implementation is optimized for the most common use case of fast and SEO-friendly first page loads, and then letting React take over from there to get the interactivity and smooth page transitions you can get with a JS framework.

So of course I had to go and experiment with making it work for that other scenario with 100% static, basic, html pages 😁

I wanted to build out a basic blog that uses markdown files for the blog posts. So the api side of the blog will not read from a database, but rather read files from the file system. This wouldn't even work on Netlify or other JAM-Stack focused hosting providers, because we don't bundle any files like that into the lambdas/serverless functions. So there's nothing to read from the filesystem. But still, it's a pretty common use case to have plog posts as markdown files and then render html pages from those. And locally it works just fine, so let's start there!

I made a /blog folder at the root of my project and placed three markdown files in there

I then went and added a BlogPost model to schema.prisma. You might wonder why I'm adding a database model, when I said I wasn't going to read from a database. But that's only because a lot of Redwood's generators needs a model in there to work with. When everything is generated we can just remove it. We never need to actually migrate our database to include it.

model BlogPost {
  slug      String    @id
  author    String
  createdAt DateTime
  updatedAt DateTime?
  title     String
  body      String
}

With the model in place I could run yarn rw g sdl BlogPost, yarn rw g cell BlogPost and yarn rw g page BlogPost. I installed front-matter on my api side to parse the .md files, and markdown-it on the web side to render the markdown to html.

The blogPosts service reads a markdown file (with frontmatter metadata) from the file system, and returns it. Redwood takes that and sends it to the web side using graphql. A Cell gets it, renders the title, the author etc to specific elements, and puts the markdown formatted body into a single <section>

I also added a layout with a cell that fetches all the blog posts to build out a navigation menu.

This all works great in my dev environment, both as it is like a normal client/sever React app. But also as pre-rendered pages. Trying to deploy it though, and you'll run into a few problems. This is still just a canary release, so deploying to Netlify (and Vercel, maybe others) is straight up broken. We can work around that on our own though. I have an issue open for it and a PR with a potential solution that's waiting for the rest of the Redwood team to take a look at it. You can read the issue and the linked PR for more detailed background info. But the solution for now is to update your build command in netlify.toml

[build]
command = "yarn rw prisma generate && yarn rw build --verbose"

Compared to the standard build command we're generating the prisma client first, before building anything and that's the key to solving the problem. I also don't do any prisma migrations or data migrations because we don't need any of that since we're not using a database.

Now it should build and deploy successfully.

Going to the page it'll quickly flash the content you want to see, but then just show this 🙁

What gives?

There are a couple of things going on here. First the web server serves the pre-rendered html page, and as soon as the web browser has that page it will start downloading the JavaScript bundles for the page. While it's downloading you see the pre-rendered page. But then when it's finished downloading and renders the React pages and components the BlogPostCell will fire off a gql request to the api side. It'll show its loading state ("Loading...") while the gql does its roundtrip to the api side. And now we'll run into the second problem. As I said at the beginning, when deploying to a serverless environment we're not bundling any extra files with the lambdas. So when the blogPost service tries to read the markdown file on the filesystem it fails, because the files aren't there. And so some random error page is returned to the Cell and we get the Cell's error state.

So what's the solution here? Get rid of the JavaScript that replaces our perfectly fine pre-rendered page, and just keep that! We're probably going to add this functionality into Redwood itself. But not yet. So for now we'll have to hack around it (and I ❤️ it!)

To solve this we're going to use one of RW's most underrated features - its scripts! Generate a new script yarn rw g script ssg and then execute it as part of the build process

[build]
command = "yarn rw prisma generate && yarn rw build web --verbose && yarn rw exec ssg"

I also went ahead and told it to only build the web side. Because there won't be any javascript triggering any calls to the api side we don't even need it at all anymore.

The ssg script goes through all the generated html pages and does a string replace to remove all <script> tags that download the React framework code and our app-specific code.

And there we have it! Redwood as a static site generator generating purely static html pages and serving them up on Netlify. Gatsby we're coming for you! (Just kidding, we love Gatsby and all other web frameworks too 😀)

The demo page is deployed here: https://rw-static-site-generator.netlify.app. And the code for it is on GitHub: https://github.com/Tobbe/rw-static-site-generator.

Go visit the page. Navigate around and you'll notice that it does a full page reload for each page you go to. Normally this isn't all that great. But since the pages are all so small, and just basically text files on some server somewhere, it actually works out great in the end anyway! And now, turn off JavaScript and you'll notice... No difference! Because we ain't using no JS for these pages anymore 😄

Cover photo by Ricardo Gomez Angel on Unsplash

We'll be using keonik/prisma-erd-generator which is an npm package that you can use like a kind of plugin to prisma to generate an svg ER diagram of your prisma schema.

First thing you need to do is to install it. In the root of your RW project, run this command (yes, we're not specifying any workspace here):

yarn add -D prisma-erd-generator @mermaid-js/mermaid-cli

You'll then need to add a bit of config to the top of your schema file (api/prisma/schema.prisma), right below the generator config for client (generate client { ... })

generator erd {
  provider = "prisma-erd-generator"
  output   = "./ERD.svg"
}

Save the file and now you're ready to generate the diagram. Run this command from the root of your project (it can take a long time the first time you run it). (If you're trying this on a fresh project, make sure you've ran at least one migration before trying to add and run the erd generator.)

yarn rw prisma generate

That should have produced an SVG file at api/prisma/ERD.svg. Open it up (for example using your web browser) and you should see all your models and relations from your schema file.

What's great here is that whenever prisma runs generate you'll get a new diagram. So, for example, whenever you generate a new migration you'll also get an up-to-date visual representation of your database model. How great is that?!

For a basic schema like this:

model Post {
  id        Int      @id @default(autoincrement())
  title     String
  body      String
  createdAt DateTime @default(now())
  userId    Int?
  user      User?    @relation(fields: [userId], references: [id])
}

model Contact {
  id        Int      @id @default(autoincrement())
  name      String
  email     String
  message   String
  createdAt DateTime @default(now())
}

model User {
  id                  Int       @id @default(autoincrement())
  email               String    @unique
  hashedPassword      String
  salt                String
  resetToken          String?
  resetTokenExpiresAt DateTime?
  roles               String?
  posts               Post[]
}

You'll get a diagram like this:

I'm using RedwoodJS, and Redwood has its own data migration solution. But I wanted to do it all with prisma for various reasons. You should decide for your particular scenario what works best for you.

Prisma migrations are just standard .sql files, so if you can express your data migration as one or more sql statements you're good to go.

As I said, I'm using RedwoodJS, so first I edit my api/db/schema.prisma file to do my schema migration. In this case I have a "Products" model with a soft/implicit self-reference. I wanted to make that a proper Prisma one-to-many relation. The self-reference is used to model product variations. Like you have a main "Product" that's a shirt. Then you have variants for the different colors of the shirt. So you could have three variants for "red", "green" and "blue" shirts.

This is the diff for my schema.prisma-file

diff --git a/api/db/schema.prisma b/api/db/schema.prisma
index d29fc77..40ffc59 100644
--- a/api/db/schema.prisma
+++ b/api/db/schema.prisma
@@ -45,6 +45,9 @@ model Product {
   stock               Int                 @default(0)
   images              String?
   parent_v_p_id       Int?
+  parentId            String?             @db.Uuid
+  parentProduct       Product?            @relation("ParentProduct", fields: [parentId], references: [id])
+  variants            Product[]           @relation("ParentProduct")
   ean                 String?
   attrib1name         String?
   attrib1val          String?

Next up I run yarn rw prisma migrate dev --create-only. The key here is the --create-only flag. It creates a .sql file with the migration instruction in it, but doesn't actually apply it. This gives us a chance to modify the SQL statements the migration will run.

This is what the generated file looks like

-- AlterTable
ALTER TABLE "Product" ADD COLUMN     "parentId" UUID;

-- AddForeignKey
ALTER TABLE "Product" ADD FOREIGN KEY ("parentId") REFERENCES "Product"("id") ON DELETE SET NULL ON UPDATE CASCADE;

As you can see the Prisma relation is made up of three fields, but only one is actually created in the database. The parentId field in this case. I'm using the database-native datatype UUID here to match with the data type of my id field.

I mentioned I had a soft or implicit self-reference for product variants. Half of it is the parent_v_p_id field you can see in the diff output above. The other half is a variable_product_id field. I call products that have different variants for "variable" products. (And products without variants for "simple" products.) Now however I decided to just use the id that all products have as the relationship reference. No need for a specific field when I already have a perfectly good id to use.

Now that we know what we want to do, let's write the SQL for it.

I'm not great at SQL, but I do know the basics, so after a few minutes on Google I came up with this statement*

UPDATE "Product"
SET "parentId" = p.id
FROM (
  SELECT id,
         variable_product_id AS v_p_id
  FROM "Product"
) AS p
WHERE parent_v_p_id IS NOT NULL
  AND p.v_p_id = parent_v_p_id;

(* Not really true. I came up with something similar, but had to do some additional tweaks to arrive at what you see above)

Modifying the DB, and especially editing the data in it, is scary. So I wanted a way to test my SQL before making it part of the migration. This is where I turned to my db admin tool of choice, DBeaver. I'm sure TablePlus or something like that would work great as well. Just need a way to run custom SQL statements. The trick I used to be able to test my code was to wrap it in a transaction that I always roll back at the end. That way I can iterate on the sql without doing any permanent changes to the database.

BEGIN;

-- [migration statements]...

ROLLBACK;

I did have to make a few tweaks to my original SQL. When the entire transaction completed without errors I was ready to try it on my test database. So I just copied the statements between the BEGIN; and ROLLBACK; lines and replaced all the content of my migration .sql file with the copied code.

-- 20211128123650_parentproduct_relation/migration.sql

-- AlterTable
ALTER TABLE "Product"
ADD COLUMN "parentId" UUID;

-- AddForeignKey
ALTER TABLE "Product"
ADD FOREIGN KEY ("parentId")
  REFERENCES "Product"("id")
  ON DELETE SET NULL
  ON UPDATE CASCADE;

UPDATE "Product"
SET "parentId" = p.id
FROM (
  SELECT id,
         variable_product_id AS v_p_id
  FROM "Product"
) AS p
WHERE parent_v_p_id IS NOT NULL
  AND p.v_p_id = parent_v_p_id;

One final precaution to take (depending on how much you care about your test database) is to dump your database to a file. I usually do that with this command

pg_dump \
  --username=username \
  --password \
  --format=plain \
  --inserts \
  --no-owner \
  --no-privileges \
  db_name > db_name_dump_$(date +"%Y%m%dT%H%M%S").sql

It results in a big file that's slow to restore. But it's also a file that's easy read (and understand) and that will work across different versions of PostgreSQL and maybe even between different DB engines. I think it's worth the tradeoffs for when working with my test db. For backing up your prod db you might want to use something else. But now we're off on a tangent. Let's get back to data migrations...

With the migration script updated it's time to apply it. Running yarn rw prisma migrate dev again, but without any extra flags this time, will apply the migration to your test database.

After running that it's obviously a good idea to verify that everything looks alright in the database. Again DBeaver (or TablePlus etc) comes in handy. Finally you should spin up your app to make sure it runs as expected. Hopefully the db survived the migration and your data (and schema) is properly migrated.

That's it 🙂 Thanks for reading!

Cover photo by Andrey Novik on Unsplash

Looking for existing guides on how to integrate Google Maps I found mention of two React components, https://github.com/tomchentw/react-google-maps and https://github.com/fullstackreact/google-maps-react. They both seem fairly popular on npm too, with 144k and 60k weekly downloads respectively. A quick check on bundlephobia showed that google-maps-react was about 1/3 the size of react-google-maps. So even though it wasn't as popular I went with the smaller option. Only to find out their docs are outdated and their TypeScript support is broken 🙁

Turns out both project are pretty much abandoned too, with lots of open issues without any replies. So back to Google to find other options.

I then found @react-google-maps/api which is a rewrite of react-google-maps, the most popular of the packages I evaluated earlier. This rewrite is made for modern versions of React with hooks and function components 💯 Nice! To be honest their docs could use some work (like a total overhaul), but after a couple of tries I got something working!

While writing this blog post I also found https://github.com/google-map-react/google-map-react which is the most popular option I've found so far with 215k weekly downloads. But that library seems more focused on rendering your own custom markers on the map. I just wanted the default Google Maps pin, which is super easy with the package I used.

Implementation

Begin by adding the package to your app

yarn workspace web @react-google-maps/api

You'll need a Google Maps API key, so while the package is downloading and installing you can go to https://developers.google.com/maps/documentation/javascript/get-api-key and set everything up. You don't need to set up billing to use it, but you'll have an ugly "for development only" overlay and lots of watermarks on your map until you do.

When you have your API key you should go ahead and paste it in your .env file. (If you don't have a .env file, look for a .env.defaults file and create an empty .env file next to it.) I'm going to use the key name GOOGLE_MAPS_API_KEY. So place this on a row of its own

GOOGLE_MAPS_API_KEY=AIezaChaosFml69ggpW7YlQHnndx-QBesos

It should look something like the above. To be able to use it in your front-end code you also have to whitelist it in redwood.toml

[web]
  includeEnvironmentVariables = [
    'GOOGLE_MAPS_API_KEY',
  ]

After you can created a Map component to use on your About page (or wherever you want, obv).

yarn rw g component Map

Open up Map.tsx and replace everything inside with this

import { GoogleMap, Marker, useJsApiLoader } from '@react-google-maps/api'

const containerStyle = {
  width: '100%',
  height: '400px',
  marginTop: '16px',
}

const center = {
  lat: 37.419857,
  lng: -122.078827,
}

const Map = () => {
  const { isLoaded, loadError } = useJsApiLoader({
    googleMapsApiKey: process.env.GOOGLE_MAPS_API_KEY,
  })

  if (loadError) {
    // This would be a great place to show just a static image of a map as a
    // fallback option
    return <p>Error loading map</p>
  }

  return isLoaded ? (
    <GoogleMap
      zoom={10}
      center={center}
      mapContainerStyle={containerStyle}
    >
      <Marker position={center} />
    </GoogleMap>
  ) : (
    <p>Loading...</p>
  )
}

export default React.memo(Map)

It's important to note that you need to give the container some dimensions. Otherwise the map won't show up.

And now we're ready to use it on our About page

import Map from 'src/components/Map'

const AboutPage = () => {
  return (
    <div>
      <address>
        <strong>Googleplex</strong>
        <br />
        1600 Amphitheatre Pkwy
        <br />
        Mountain View
        <br />
        CA 94043
        <br />
        United States
      </address>
      <Map />
    </div>
  )
}

export default AboutPage

This is what it'll look like (it's just a screenshot, so it's not interactive like the real map would be)

Cover photo by Thomas Kinto on Unsplash

Gatsby is great! It gives you a lot of features and creative freedom. The thing is though - I didn't use barely any of it. I just want a place to dump my thoughts, and hashnode seems a great place to do that. Hopefully it'll help a bit with reach as well.

Here are three great features of hashnode that they highlight on their front page

Writing this post I can add another thing that's really nice, and that's their online editor! Even let me just copy-paste any image and it's automatically uploaded to their CDN and linked in my post 🏞

What I would like to see though, is better docs on how to migrate your existing blog to hashnode. If the docs are there, I couldn't find them. And while they have a pretty prominent link to instructions for linking your own domain to hashnode there was really no mention of what to expect once it was linked. All I got was this:

That doesn't look very nice. Some hints on what to do to prepare for linking would have been nice. Like "move your content over to us first, link your domain after", if that's indeed the case. TBD.

Update: A bit later I got this, which obviously isn't perfect, but at least it's 100x better!

I've also found an import tool

I already have my posts as .md markdown files, so hopefully the Bulk Markdown importer should be helpful (it takes a .zip-file with a single .md-file for each post). Wish me luck 🤞!

Cover photo by Robinson Greig on Unsplash

If you don't know what Dokku is, the short version is that it's like a self-hosted version of Heroku. It uses the same buildpacks, procfiles and deployment process as Heroku. There are other, more advanced, options as well, if the Heroku stuff is too limiting for you, but I won't go into that here. Since it's like Heroku, that also means you could host your database in Dokku if you wanted. But that's also not covered by this guide.

A big shoutout to Brent Anderson for his gist on getting RW running on Heroku. I could not have written this guide without that gist.

Beware that it's you, yourself, who is responsible for keeping your server secure and up-to-date. You have to figure out backups. What do you do if your page becomes super popular? How do you handle scaling? There are a thousand and one reasons to not self-host. Begin has a whole page dedicated to why you shouldn't do it. It's a pretty fun, and eye-opening read. Have a look: https://begin.com/learn/shit-youre-not-doing-with-begin

With that out of the way: let's get started!

To follow along you'll need an Ubuntu 20.04 box (or a recent version of Debian) where you have root access. The server needs to have at least 1 GB of RAM. You'll also need a domain you can configure DNS for. And finally, a suitable Redwood project to deploy. I have a small cloud server at Hetzner and a domain that I bought through Porkbun. If you have a DigitalOcean droplet cloud server, or a Microsoft Azure server there are specific guides on Dokku's web page 1 2

First thing to do is configure your domain to point to the IP address of your server (you need a wildcard record). Make sure it's working by SSHing into your server. If this is a freshly installed OS, create a new user (do not name it "dokku") and make sure it's allowed to use the sudo command. From now on, use this user to execute all commands.

Install Dokku

wget https://raw.githubusercontent.com/dokku/dokku/v0.24.3/bootstrap.sh
sudo DOKKU_TAG=v0.24.3 bash bootstrap.sh

Go to yourdomain.com and finish the setup in your browser.

Create a new Dokku application (replace my-redwood-app with the actual name of your app).

dokku apps:create my-redwood-app

If you have a database connected to your app you'll want to set the DATABASE_URL environment variable.

dokku config:set my-redwood-app DATABASE_URL=postgresql://asldfjsldf

Now it's time to prepare your Redwood app for deploying to Dokku. You need to do two things:

1. Set apiProxyPath = "/api" in redwood.toml
2. Create a new file in the root of your project called .buildpacks. Add this content to that file

   https://github.com/tobbe/dokku-buildpack-redwood-init.git
   https://github.com/heroku/heroku-buildpack-nodejs.git
   https://github.com/heroku/heroku-buildpack-nginx.git
   https://github.com/tobbe/dokku-buildpack-redwood-finish.git
   

Now your app is ready. For the next step you'll need to have an ssh agent running, with your key loaded. If you're on Windows you might have to do this manually. These two commands worked for me in git-bash.

eval `ssh-agent -s`
ssh-add ~/.ssh/id_rsa

Now you can add dokku as a new remote to git, and push to deploy! 🚀

git remote add dokku dokku@yourdomain.com:my-redwood-app
git push dokku main

This will take a few minutes as it downloads and installs node.js, nginx, redwood and your other project dependencies etc. But in the end, when it's ready, you should be able to go to my-redwood-app.yourdomain.com and see your site live in your browser!

If you made it this far, and it works, congratulations! 🎉🏁

I bet a few of you who read this wonder what those github hosted buildpacks do that we added to the .buildpacks file. The first one is written by me, and it adds a few files to your Redwood app. Then we add nodejs and nginx. One of the files added in the first step is a config file for nginx. That's why the "redwood-init" buildpack needs to run first. Finally there's another Redwood specific buildpack that installs a few needed packages, builds Redwood and finally starts nginx and Redwood's api server.

🔒 Now that your Redwood app is live, I highly recommend setting up SSL for it, to make it more secure. I'm just going to link to Dokku's SSL guide. It's really easy to get Let's Encrypt setup.

(Header photo by Alex Duffy on Unsplash)

This is what we'll build

Let's start with a new Redwood project and generate a signup page to add our form to.

yarn create redwood-app signup-example
cd signup-example/
yarn rw g page signup

Open up SignupPage.js in your code editor of choice and change it to look like this

import {
  Form,
  TextField,
  EmailField,
  PasswordField,
  Label,
  Submit,
} from "@redwoodjs/forms";

const SignupPage = () => {
  const onSubmit = (data) => {
    console.log("Submitted form with data", data);
  };

  return (
    <Form onSubmit={onSubmit}>
      <Label name="name">Name:</Label>
      <TextField name="name" />

      <Label name="email">Email:</Label>
      <EmailField name="email" />

      <Label name="password">Password:</Label>
      <PasswordField name="password" />

      <Label name="password_confirm">Confirm Password:</Label>
      <PasswordField name="password_confirm" />

      <Submit>Submit</Submit>
    </Form>
  );
};

export default SignupPage;

And just to make it display a tiny bit better we'll add some CSS rules to index.css

label,
button,
input {
  display: block;
}

label,
button {
  margin-top: 1em;
}

span {
  color: red;
}

Now you have a basic form you can use to let users signup on your website. Enter some data and press Submit and you'll see everything printed to the web browser console.

Next thing I wanted to do was to add some validation to the form. We want to make all fields required.

For this we can use the regular html5 required attribute, like so

return (
  <Form onSubmit={onSubmit}>
    <Label name="name">Name:</Label>
    <TextField name="name" required />

    <Label name="email">Email:</Label>
    <EmailField name="email" required />

    <Label name="password">Password:</Label>
    <PasswordField name="password" required />

    <Label name="password_confirm">Confirm Password:</Label>
    <PasswordField name="password_confirm" required />

    <Submit>Submit</Submit>
  </Form>
);

Now, if you try to submit the form without filling in all the fields we'll see something like this

But Redwood can do much better than that! Let's switch over to @redwood/forms' custom validation. One gotcha here is that you have to switch all fields over, you can't just add custom validation to one field, to try it out. If you do, the form will get confused. So you have to pick one way or the other for all your form fields.

Change your code to look like this to get "required" validation with custom messages.

return (
  <Form onSubmit={onSubmit} noValidate validation={{ mode: "onBlur" }}>
    <Label name="name">Name:</Label>
    <TextField name="name" validation={{ required: "Name is required" }} />
    <FieldError name="name" />

    <Label name="email">Email:</Label>
    <EmailField name="email" validation={{ required: "Email is requried" }} />
    <FieldError name="email" />

    <Label name="password">Password:</Label>
    <PasswordField
      name="password"
      validation={{
        required: "You must choose a password",
      }}
    />
    <FieldError name="password" />

    <Label name="password_confirm">Confirm Password:</Label>
    <PasswordField
      name="password_confirm"
      validation={{ required: "You have to confirm your password" }}
    />
    <FieldError name="password_confirm" />

    <Submit>Submit</Submit>
  </Form>
);

As you can see we removed required and added validation instead. required is a regular html attribute, and validation is a prop from @redwoodjs/forms. As I mentioned earlier @redwoodjs/forms uses react-hook-form under the hood, and that validation syntax we used here would have looked something like this if done with r-h-f instead: ref={register({ required: 'Name is required' })}. This can be good to know when reading the r-h-f docs to try to figure out how to do something more advanced. We also added noValidate to the form, because we don't want the html5 validation now that we've added our own. Also configured to validation to trigger on blur.

The email is really important, so I wanted to add a bit more strict verification for that field. It's obviously not bullet-proof, but it's better than nothing.

<EmailField
  name="email"
  validation={{
    required: 'Email is required',
    pattern: {
      value: /[^@]+@[^.]+\..{2,}/,
      message: 'Please enter a valid email address',
    },
  }}
/>

I decided to keep my password validation rules really simple. All I require is that the passwords are at least 10 characters long.

<PasswordField
  name="password"
  validation={{
    required: "Password is required",
    minLength: {
      value: 10,
      message: "Password must be at least 10 characters long",
    },
  }}
/>

The final thing I wanted to add is probably the most interesting and that's validation to make sure "Password" and "Confirm Password" matches. For this we need to use the watch method from react-hook-form. It's available as part of what you get back from the useForm() hook.

These are the parts you need to set it up.

First a couple of new imports.

import { useRef } from 'react'
import { useForm } from 'react-hook-form'

Here we use the imports from above. useForm() is from react-hook-form and gives us full control over the form. Normally @redwoodjs/forms set this up for us, but now we need our own, to set up a watch on the password. We store the watched value in a ref we get from React's useRef(). When you add a watch you're going to introduce more re-renders. But only when that watched field changes, not when any other field in the form changes.

const formMethods = useForm()
const password = useRef()
password.current = formMethods.watch('password', '')

As I said, we're now using our own formMethods, so we have to let Redwood know to use that one, and not create one of its own.

<Form
  onSubmit={onSubmit}
  noValidate
  validation={{ mode: 'onBlur' }}
  formMethods={formMethods}
>

Finally, for the validation, we add a custom validate method to the validation object. These custom validate methods receive the current field value as a parameter, and should return true when the field is valid, and false or a string when the field is not valid. The string, if that's what you return, will be the error message displayed for that field.

You can read more about validate and all other validation possibilities in the r-h-f docs: https://react-hook-form.com/api#register

<PasswordField
  name="password_confirm"
  validation={{
    required: 'You must confirm your password',
    validate: (value) =>
      value === password.current || 'Passwords must match',
  }}
/>

There is one more thing I wanted to add. Since this is a signup page where the user should pick a new password I want the browser to show the password suggestion box.

The browser will try to figure out by context if it should show that box or not. And often it does the right thing. But there's no need for us to leave it to chance. If we tell the browser this is a new-password field it knows to show that ui. It's also helpful to tell the browser what field is the user name. We do this by setting autoComplete="new-password" and autoComplete="username" respectivly. Read more about these and many more options at MDN

So this is the password field, notice the autoComplete attribute at the end

<PasswordField
  name="password"
  validation={{
    required: 'Password is required',
    minLength: {
      value: 10,
      message: 'Password must be at least 10 characters long',
    },
  }}
  autoComplete="new-password"
/>

Wrapping it all up, here's the full SignupPage.js

import { useRef } from 'react'
import { useForm } from 'react-hook-form'
import {
  Form,
  TextField,
  EmailField,
  PasswordField,
  Label,
  Submit,
  FieldError,
} from '@redwoodjs/forms'

const SignupPage = () => {
  const formMethods = useForm()
  const password = useRef()
  password.current = formMethods.watch('password', '')

  const onSubmit = (data) => {
    console.log('Submitted form with data', data)
  }

  return (
    <Form
      onSubmit={onSubmit}
      noValidate
      validation={{ mode: 'onBlur' }}
      formMethods={formMethods}
    >
      <Label name="name">Name:</Label>
      <TextField name="name" validation={{ required: 'Name is required' }} />
      <FieldError name="name" />

      <Label name="email">Email:</Label>
      <EmailField
        name="email"
        validation={{
          required: 'Email required',
          pattern: {
            value: /[^@]+@[^.]+\..{2,}/,
            message: 'Please enter a valid email address',
          },
        }}
        autoComplete="username"
      />
      <FieldError name="email" />

      <Label name="password">Password:</Label>
      <PasswordField
        name="password"
        validation={{
          required: 'Password is required',
          minLength: {
            value: 10,
            message: 'Password must be at least 10 characters long',
          },
        }}
        autoComplete="new-password"
      />
      <FieldError name="password" />

      <Label name="password_confirm">Confirm Password:</Label>
      <PasswordField
        name="password_confirm"
        validation={{
          required: 'You must confirm your password',
          validate: (value) =>
            value === password.current || 'Passwords must match',
        }}
        autoComplete="new-password"
      />
      <FieldError name="password_confirm" />

      <Submit>Submit</Submit>
    </Form>
  )
}

export default SignupPage

(Header photo by Paulius Dragunas on Unsplash)

The way the preview customization works is you tell CMS what collection to use the custom preview for, and you give it a React component to use. You can only write old-school class components, and you have to use the createClass and h helper functions provided by NetlifyCMS.

Below is an example that creates a DocPreview React component and tells CMS to use that whenever it's showing a preview for a document in the "pages" category. The render() function uses the widgetFor() helper function (read more in the NetifyCMS docs) to get the default widget used to display the 'body' field, and nothing else. By default the preview will show all fields, like "title" etc, but here we're stripping those out.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Netlify CMS Example</title>
  </head>
  <body>
    <script src="https://unpkg.com/netlify-cms@^2.10.0/dist/netlify-cms.js"></script>
    <script>
      const DocPreview = createClass({
        render: function() {
          return this.props.widgetFor('body');
        },
      });

      window.CMS.registerPreviewTemplate("pages", DocPreview);
    </script>
  </body>
</html>

Another thing we can do is take the plain text from the "body" field and dump it in a <div> with the h() function, which is basically just an alias for React's createElement().

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Netlify CMS Example</title>
  </head>
  <body>
    <script src="https://unpkg.com/netlify-cms@^2.10.0/dist/netlify-cms.js"></script>
    <script>
      const DocPreview = createClass({
        render: function() {
          const bodyText = this.props.entry.getIn(["data", "body"]);

          return h("div", {}, bodyText);
        },
      });

      window.CMS.registerPreviewTemplate("pages", DocPreview);
    </script>
  </body>
</html>

Since we're writing all the code in a basic index.html file without any kind of build tooling to support us we can't just do npm install and then include-ing the package. All we have to work with is the <script> tag. So whatever we want to use has to be in AMD or UMD module format. Thankfully react-markdown provides an umd build we can use straight from unpkg.com.

So just put https://unpkg.com/react-markdown@5.0.3/react-markdown.min.js in a <script> tag, and you'll have window.ReactMarkdown available, which is a React component you can use inside your custom render method.

With this you should be back to something that works the same as when we used widgetFor() except now we're using our own Markdown renderer, and not Netlify CMS's.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Netlify CMS Example</title>
  </head>
  <body>
    <script src="https://unpkg.com/netlify-cms@^2.10.0/dist/netlify-cms.js"></script>
    <script src="https://unpkg.com/react-markdown@5.0.3/react-markdown.min.js"></script>
    <script>
      const DocPreview = createClass({
        render: function() {
          const bodyText = this.props.entry.getIn(["data", "body"]);

          return h(window.ReactMarkdown, {}, bodyText);
        },
      });

      window.CMS.registerPreviewTemplate("pages", DocPreview);
    </script>
  </body>
</html>

The final step is to add support for Github Flavored Markdown, so we can do tables, strikethrough text and other things. The window.ReactMarkdown component takes a plugin prop that lets you augment it with different plugins, one which is remark-gfm. Unfortunately that package doesn't have a UMD build for us to use, it's only available in CJS (CommonJS) format. There is a tool called Browserify that takes CJS modules and converts them to modules that can be used by the browser. But it's a command line tool that you'd add to your tooling. Which, we don't have here. Thankfully someone turned it into a web service we can use! https://wzrd.in takes any npm package and tries to run it through Browserify and serves up the built package for you. First time it's run it takes a while, but after that it's cached and then it's much faster. Here's the link for the Browserified version of remark-gfm: https://wzrd.in/standalone/remark-gfm@1.0.0. Add that to a script tag and you'll have the plugin available at window.remarkGfm. Add that as a plugin to react-markdown and you're done!

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Netlify CMS Example</title>
  </head>
  <body>
    <script src="https://unpkg.com/netlify-cms@^2.10.0/dist/netlify-cms.js"></script>
    <script src="https://unpkg.com/react-markdown@5.0.3/react-markdown.min.js"></script>
    <script src="https://wzrd.in/standalone/remark-gfm@1.0.0"></script>
    <script>
      const DocPreview = createClass({
        render: function() {
          const bodyText = this.props.entry.getIn(["data", "body"]);

          return h(window.ReactMarkdown, { plugins: [window.remarkGfm] }, bodyText);
        },
      });

      window.CMS.registerPreviewTemplate("pages", DocPreview);
    </script>
  </body>
</html>

Try writing some markdown jazzed up with a bit of gfm syntax to make sure everything works.

That's it. Hope this guide was helpful!

(Header photo by Richy Great on Unsplash)

By default when you create a new RedwoodJS app this is what you get in your index.js file:

ReactDOM.render(
  <FatalErrorBoundary page={FatalErrorPage}>
    <AuthProvider client={netlifyIdentity} type="netlify">
      <RedwoodProvider>
        <Routes />
      </RedwoodProvider>
    </AuthProvider>
  </FatalErrorBoundary>,
  document.getElementById('redwood-app')
)

The interesting bit is <RedwoodProvider>. Looking at the source for Redwood we see this:

export { RedwoodApolloProvider as RedwoodProvider } from './components/RedwoodApolloProvider'

and this:

import {
  ApolloProvider,
  ApolloClientOptions,
  ApolloClient,
  InMemoryCache,
  useQuery,
  useMutation,
} from '@apollo/client'

// Other imports...

const ApolloProviderWithFetchConfig: React.FunctionComponent<{
  config?: Omit<ApolloClientOptions<InMemoryCache>, 'cache'>
}> = ({ config = {}, children }) => {
  const { uri, headers } = useFetchConfig()

  const client = new ApolloClient({
    cache: new InMemoryCache(),
    uri,
    headers,
    ...config,
  })

  return <ApolloProvider client={client}>{children}</ApolloProvider>
}

export const RedwoodApolloProvider: React.FunctionComponent<{
  graphQLClientConfig?: Omit<ApolloClientOptions<InMemoryCache>, 'cache'>
  useAuth: () => AuthContextInterface
}> = ({ graphQLClientConfig, useAuth, children }) => {
  return (
    <FetchConfigProvider useAuth={useAuth}>
      <ApolloProviderWithFetchConfig config={graphQLClientConfig}>
        <GraphQLHooksProvider useQuery={useQuery} useMutation={useMutation}>
          <FlashProvider>{children}</FlashProvider>
        </GraphQLHooksProvider>
      </ApolloProviderWithFetchConfig>
    </FetchConfigProvider>
  )
}

So <RedwoodProvider> is a renamed export of <RedwoodApolloProvider> that wrapps the <ApolloProvider> context around its children, and passes useQuery and useMutation from @apollo/client to <GraphQLHooksProvider>.

The new powerful thing is that we can remove <RedwoodProvider> from our code and do what it does on our own instead — and that gives us the ability to pass in other useQuery and useMutation hooks from some other GraphQL client. For Apollo Client it's super easy. (It's almost as if Redwood was built for usage with Apollo Client 😜) All you have to do is import useQuery and useMutation and pass them straight into <GraphQLHooksProvider>. For any other graphql client you are probably going to have to write some adapter code to make it all work.

The other thing we need to do is to create our graphql client. And the client will need to know what headers to send and what url to talk to. For this we have the useFetchConfig() hook. Again, it's super straightforward to use with Apollo Client, but should be fairly easy to use with your client of choice as well.

This is an example of how it can be done when wiring up graphql-hooks

const useQueryAdapter = (query, options) => {
  return useQuery(print(query), options)
}

const useMutationAdapter = (query, options) => {
  return useMutation(print(query), options)
}

const GraphqlHooksClientProvider = ({ children }) => {
  const { uri: url, headers } = useFetchConfig()

  const client = new GraphQLClient({ url, headers })

  return (
    <ClientContext.Provider value={client}>{children}</ClientContext.Provider>
  )
}

ReactDOM.render(
  <FatalErrorBoundary page={FatalErrorPage}>
    <AuthProvider client={netlifyIdentity} type="netlify">
      <FetchConfigProvider>
        <GraphqlHooksClientProvider>
          <GraphQLHooksProvider
            useQuery={useQueryAdapter}
            useMutation={useMutationAdapter}
          >
            <FlashProvider>
              <Routes />
            </FlashProvider>
          </GraphQLHooksProvider>
        </GraphqlHooksClientProvider>
      </FetchConfigProvider>
    </AuthProvider>
  </FatalErrorBoundary>,
  document.getElementById('redwood-app')
)

The adaptors for the hooks are simple. Only change we had to do was to transform the graphql queries that come as GQL ASTs in to plain strings. We use the print function for this. Setting up the client using useFetchConfig() is also easy, just have to rename uri to url for graphql-hooks to be happy.

You can see a full implementation in this GitHub repo: https://github.com/Tobbe/redwood-graphql-hooks (But there really isn't much more to it than what I've shown here.)

So, why do we have to let Redwood know about our useQuery and useMutation hooks in the first place? useQuery is used internally by Redwood with Cells, in its withCellHOC. useMutation technically wouldn't be necessary. But having it there allows the generators to generate code that runs and is valid. Without it, generated code like this would never be valid: import { useMutation, useFlash } from '@redwoodjs/web'. (That line is from the EditNameCell.js.template file.)

(Header photo by Armand Khoury on Unsplash)

Here's a step-by-step guide or tutorial on how I did it.

AWS S3 Bucket

The first step is going to be to set up our storage. Go to https://aws.amazon.com/ and sign in to the AWS Management Console (or create an account if you don't have one already).

Choose to sign in as "Root user" and then you'll find S3 under "Storage" to the left. Here's a direct link that you might be able to use https://s3.console.aws.amazon.com/s3/home?region=us-east-1. Click "Create bucket" and give it a unique name. All other options can be left with their default values, so just scroll down and click "Create bucket"

Now that you have a new bucket created, it's time to set up a user with a policy that lets it access the files in the bucket. In the upper left corner of the screen you click on "Services" and then you can search for "iam" and click the first and only result to go to "Identity and Access Management".

Click "Policies" in the left menu and then the blue "Create policy" button up top.

When creating your new policy you should select the "S3" service, choose the "GetObject" Read action, add the ARN (Amazon Resource Name) for the files you want to give access to by typing in the name of your newly created bucket and clicking "Any" on the object name to allow access to all files in the bucket, and finally you can leave the "Request conditions" as it is.

This is what it looked like for me when I created the policy

Click "Review policy" (bottom right), give your policy a name, like "example-secure-bucket-tlundberg-com-policy" and finally click the blue "Create policy" button.

Now, with the policy created, we'll go ahead and create a user that we'll connect this policy to. So click "Users" in the left menu and the the blue "Add user" button. Give the user a name, like "example-secure-bucket-tlundberg-com-user" and select the "Programatic access" access type. This will let you use this user with the AWS SDK and APIs.

Now click "Next: Permissions", and on the new page you chose "Attach existing policies directly" (it's the last box in the row up top). Filter the list of policies to find the one you created previously and click the checkbox next to it.

This was the last step where we had to do anything, so just click "Next: Tags", "Next: Review" and finally "Create user".

Now it's important you save the secret access key for this user, because you will not be able to find it again. Either download the .csv file, or copy/paste the values to somewhere safe. After you've saved the credentials you can close the "Add user" wizard. Should you lose the Access key ID and/or the Access Secret you can come back here to generate new ones.

You're finally done with AWS and it's time to move on to RedwoodJS stuff.

RedwoodJS Function

Create a new Redwood project if you don't have one already, and add a new file in api/src/functions/. I called mine s3download.js. Add this code to the file

const AWS = require('aws-sdk')

export const handler = async () => {
  const s3 = new AWS.S3({
    accessKeyId: process.env.S3_KEY_ID,
    secretAccessKey: process.env.S3_SECRET,
    region: 'us-east-1',
  })

  try {
    const s3Result = await s3
      .getObject({
        Bucket: process.env.S3_BUCKET,
        Key: 'my_s3_file.txt',
      })
      .promise()

    const s3File = s3Result.Body.toString('utf-8')

    console.log('file contents', s3File)
    return { statusCode: 200, body: 'File downloaded successfully' }
  } catch (err) {
    console.error(err)
    return { statusCode: 500, body: err.message }
  }
}

We have to install the aws-sdk package to use this function. Do that by running yarn workspace api add aws-sdk.

As you can see the function uses three environment variables. Under no circumstances should you put your bucket credential directly in your source code, because it will (probably) be pushed to GitHub, where other people could see it. Even if it's a private repo it's just yet anohter place your credentials could be compromised. So instead we use environment variables. Let's add them to the .env file

S3_KEY_ID=KYSA9EKSDFHCK88194UK
S3_SECRET=dsli5lsi92klsjdf120sdfGsiSDDKSKS3sdflkjS
S3_BUCKET=example-secure-bucket-tlundberg-com

Those are just made-up values to show you what it should look like, use your values instead. Also make sure the region in the code matches the region you have your bucket in.

Before we can test this we need to upload the my_s3_file.txt file to the s3 bucket. Easiest is to just drag-and-drop it in the AWS web interface. So go ahead and do that.

It's finally time to try it all out! Run yarn rw dev and you should be able to access your function at http://localhost:8911/s3download. You should see the message "File downloaded successfully" in your browser, and if you switch over to your console you should see the content of the file.

Did it work? Congratulations! All the AWS setup is not easy. Thankfully it's pretty easy to use the SDK once everything is set up correctly.

Thanks for reading!

(Header photo by Jessica Johnston on Unsplash)

The screenshot above shows what we're building. See the pagination at the bottom? The styling is up to you to fix.

So you have a blog, and probably only a few short posts. But as the blog grows bigger you'll soon need to paginate all your posts. So, go ahead and create a bunch of posts to make this pagination worthwhile. We'll display five posts per page, so begin with creating at least six posts, to get two pages.

We'll begin by updating the SDL. To our Query type a new query is added to get just a single page of posts. We'll pass in the page we want, and when returning the result we'll also include the total number of posts as that'll be needed when building our pagination component.

# api/src/graphql/posts.sdl.js

type PostPage {
  posts: [Post!]!
  count: Int!
}

type Query {
  postPage(page: Int): PostPage
  posts: [Post!]!
  post(id: Int!): Post!
}

You might have noticed that we made the page optional. That's because we want to be able to default to the first page if no page is provided.

Now we need to add a resolver for this new query to our posts service.

// api/src/services/posts/posts.js

const POSTS_PER_PAGE = 5

export const postPage = ({ page = 1 }) => {
  const offset = (page - 1) * POSTS_PER_PAGE

  return {
    posts: db.post.findMany({
      take: POSTS_PER_PAGE,
      skip: offset,
      orderBy: { createdAt: 'desc' },
    }),
    count: db.post.count(),
  }
}

So now we can make a GraphQL request (using Apollo) for a specific page of our blog posts. And the resolver we just updated will use Prisma to fetch the correct posts from our database.

With these updates to the API side of things done, it's time to move over to the web side. It's the BlogPostsCell component that makes the gql query to display the list of blog posts on the HomePage of the blog, so let's update that query.

// web/src/components/BlogPostsCell/BlogPostsCell.js

export const QUERY = gql`
  query BlogPostsQuery($page: Int) {
    postPage(page: $page) {
      posts {
        id
        title
        body
        createdAt
      }
      count
    }
  }
`

The Success component in the same file also needs a bit of an update to handle the new gql query result structure.

// web/src/components/BlogPostsCell/BlogPostsCell.js

export const Success = ({ postPage }) => {
  return postPage.posts.map((post) => <BlogPost key={post.id} post={post} />)
}

Now we need a way to pass a value for the page parameter to the query. To do that we'll take advantage of a little RedwoodJS magic. Remember from the tutorial how you made the post id part of the route path (<Route path="/blog-post/{id:Int}" page={BlogPostPage} name="blogPost" />) and that id was then sent as a prop to the BlogPostPage component? We'll do something similar here for the page number, but instead of making it a part of the url path, we'll make it a url query string. These, too, are magically passed as a prop to the relevant page component. And you don't even have to update the route to make it work! Let's update HomePage.js to handle the prop.

// web/src/pages/HomePage/HomePage.js

const HomePage = ({ page = 1 }) => {
  return (
    <BlogLayout>
      <BlogPostsCell page={page} />
    </BlogLayout>
  )
}

So now if someone navigates to https://awesomeredwoodjsblog.com?page=2 (and the blog was actually hosted on awesomeredwoodjsblog.com), then HomePage would have its page prop set to "2", and we then pass that value along to BlogPostsCell. If no ?page= query parameter is provided page will default to 1

Going back to BlogPostsCell there is one me thing to add before the query parameter work.

// web/src/components/BlogPostsCell/BlogPostsCell.js

export const beforeQuery = ({ page }) => {
  page = page ? parseInt(page, 10) : 1

  return { variables: { page } }
}

The query parameter is passed to the component as a string, so we need to parse it into a number.

If you run the project with yarn rw dev on the default port 8910 you can now go to http://localhost:8910 and you should only see the first five posts. Change the URL to http://localhost:8910?page=2 and you should see the next five posts (if you have that many, if you only have six posts total you should now see just one post).

The final thing to add is a page selector, or pagination component, to the end of the list of posts to be able to click and jump between the different pages.

Generate a new component with yarn rw g component Pagination.

// web/src/components/Pagination/Pagination.js

import { Link, routes } from '@redwoodjs/router'

const POSTS_PER_PAGE = 5

const Pagination = ({ count }) => {
  const items = []

  for (let i = 0; i < Math.ceil(count / POSTS_PER_PAGE); i++) {
    items.push(
      <li key={i}>
        <Link to={routes.home({ page: i + 1 })}>
          {i + 1}
        </Link>
      </li>
    )
  }

  return (
    <>
      <h2>Pagination</h2>
      <ul>{items}</ul>
    </>
  )
}

export default Pagination

Keeping with the theme of the official RedwoodJS tutorial we're not adding any css, but if you wanted the pagination to look a little nicer it'd be easy to remove the bullets from that list, and make it horizontal instead of vertical.

Finally let's add this new component to the end of BlogPostsCell. Don't forget to import it at the top as well.

// web/src/components/BlogPostsCell/BlogPostsCell.js

import Pagination from 'src/components/Pagination'

// ...

export const Success = ({ postPage }) => {
  return (
    <>
      {postPage.posts.map((post) => <BlogPost key={post.id} post={post} />)}

      <Pagination count={postPage.count} />
    </>
  )
}

And there you have it! You have now added a functioning, but somewhat ugly, pagination to your redwood blog. One techincal limitation to the current implementation is that it doesn't handle too many pages very gracefully. Just imagine what that list of pages would look like if you had 100 pages! I'll write another blog post in the future with a more fully featured pagination component.

Most of the code in this tutorial was copy/pasted from the "Hammer Blog" RedwoodJS example

If you want to learn more about pagination with Prisma and Apollo they both have excelent docs on the topic. https://www.prisma.io/docs/reference/tools-and-interfaces/prisma-client/pagination https://www.apollographql.com/docs/react/data/pagination/

Running all the tests takes a little while however, so when working on a specific feature it's somethimes helpful to be able to just run the tests for a specific package, or even just a specific test. So that's what I want to show here.

Let's say for example you want to run the tests for the cli package. First, go in to the cli directory; cd packages/cli. Then run all the tests by issuing yarn test (or yarn jest).

For more granularity you can run a single test-file by passing the path to it to jest. Here's the command and example output from running the page generator tests.

$ yarn test src/commands/generate/page/__tests__/page.test.js
yarn run v1.22.4
$ jest src/commands/generate/page/__tests__/page.test.js
 FAIL  src/commands/generate/page/__tests__/page.test.js (5.467s)
  √ returns exactly 2 files (4ms)
  × creates a page component (4ms)
  × creates a page test
  × creates a page component (1ms)
  × creates a page test (2ms)
  √ creates a single-word route name (1ms)
  √ creates a camelCase route name for multiple word names (1ms)
  √ creates a path equal to passed path

To run just one of the testcases, open up the test-file in your code editor, find the test definition and add .only to it. E.g. changing it from test('creates a page test', () => { to test.only('creates a page test', () => {. Run the test file again, and only the selected test case will execute. You can add .only to however many tests you like, and all the selected tests will run. The other tests in the file will be skipped.

Example output when selecting three testcases to run

$ yarn test src/commands/generate/page/__tests__/page.test.js
yarn run v1.22.4
$ jest src/commands/generate/page/__tests__/page.test.js
 PASS  src/commands/generate/page/__tests__/page.test.js (5.598s)
  √ creates a page test (3ms)
  √ creates a page test (1ms)
  √ creates a page component with a plural word for name
  ○ skipped returns exactly 2 files
  ○ skipped creates a page component
  ○ skipped creates a page component
  ○ skipped creates a single-word route name
  ○ skipped creates a camelCase route name for multiple word names
  ○ skipped creates a path equal to passed path

Test Suites: 1 passed, 1 total
Tests:       6 skipped, 3 passed, 9 total
Snapshots:   0 total
Time:        7.306s
Ran all test suites matching /src\\commands\\generate\\page\\__tests__\\page.test.js/i.
Done in 9.28s.

If you didn't already know, Git for Windows and its Git Bash environment is built using msys2, but it doesn't include all the binaries from that project. One of the binaries that exists, but that isn't included, is rsync. So what we need to do is to download the msys2 rsync binary, and place it somewhere Git Bash can find it.

1. Go to http://repo.msys2.org/msys/x86_64/ and download the latest version of rsync (not rsync2). At the time of this writing that is rsync-3.1.3-1-x86_64.pkg.tar.xz
2. Extract the downloaded archive. I'm using Total Commander with a .xz plugin, but 7-zip is also a great option. Download and install from https://www.7-zip.org/ if you need to.
3. Copy the contents of the extracted archive (sub-folders and all) to where you have Git for Windows installed. For me that's C:\Program Files\Git\. (The archive contains a \usr folder, and so does the git installation directory. What you want is for everything inside of the \usr folder in the archive to end up in the \usr folder in the git installation directory, ultimately ending up with, among other files, C:\Program Files\Git\usr\bin\rsync.exe)

That's it. You now have rsync installed. You can test your installation by opening up a Git Bash command line window and running rsync --version. You should see it print out version information.

Now, if you want to use rsync from the Windows Command Prompt, or from PowerShell, there is one more step.

Create a new .bat file with the following content (adjust the path to match your environment)

"C:\Program Files\Git\usr\bin\rsync.exe" %*

Name the file rsync.bat and place it somewhere in your %PATH%. I placed mine in C:\Windows\. Press Win + R and enter cmd. In the Command Prompt window that you just launched, enter rsync --version and it will find your .bat-file and run it, passing all arguments (that's what %* does in the command above) off to your newly installed rsync.exe

The first three steps above are based on https://serverfault.com/questions/310337/using-rsync-from-msysgit-for-binary-files/872557#872557 where you can also find instructions for setting up Pageant for SSH, if that's something you need.

I hope this short tutorial was useful to you. Happy rsyncing!

to what you see here right now. Quite the improvement, if I may say so myself!

For nostalgia's sake, I'll include that same link to Tetris right here. Play Tetris

So, what should I do with this new blog of mine? The idea is like most every other blog on the Internet to just have a braindump. Whenever I find something interesting, or make something worth sharing, I will put it here. The first thing I want to share is how I got this page to where it is right now. So that will be my next blog post. Other ideas I have for the future is to write "addons" or "extensions" or whatever you want to call them, to the main RedwoodJS tutorial. Something that picks up where that tutorial ends.