Lab 4 - Setting up for full-stack development

This lab is similar to lab 0 in that there are a lot of steps. Unfortunately, though there might be more steps and hiccups than with lab0. I will try to do my best to outline this process, but this is the first time that I've written this lab, so make sure that if you have any questions, you reach out on discord. With HTML, the servers merely hosted the files, our browsers would execute the code. With PHP (and other server-side technologies), the server will execute the hosted files. Having a server execute any instructions that we choose poses a much greater security risk for cloud-hosting companies. There are also fewer companies willing to just run any code for free (and it also becomes more expensive to maintain as well). It turns out that our existing pipeline with Cloudflare no longer works, as we need access to a server that allows us to process information as well as a place to store files.

Similarly using VSCode would result in us having a lot of extra extensions to get something working.

Because of all the changes, I'm going to have us switch everything, from IDE down to the hosting provider.

You will need potentially 4GB of disk space and at least a machine with 4GB of RAM for all of these tools that we will be using.

If this is an issue for you, then please let me know, but I will be mostly referring you to other articles for installing XAMPP and looking for other lightweight tools. The reason that I think this route will go better is merely for the amount of support and the ease of development once we have everything up and running. This also gives you a good experience with multiple toolchains and environments.

Let's get started!

There are four phases to this project

Because this document is long, I've provided jump links to many of the different phases and sub-phases in the lab. Clicking on any of these below will jump you to the appropriate section.

1. Installing Docker
2. Learning to wrangle Docker
3. Setting up PHPStorm
4. Setting up our Deployment Pipeline

Phase 1: Install Docker (opens in a new tab)

High-Level Overview for Phase 1

0. (Windows) Check your Windows version by going to Settings->System->About and check the Edition. If you have a version of Windows Home, then you will not install docker desktop, but follow these instructions (opens in a new tab) instead!
1. Open up your Terminal (or PowerShell if you don't have it) as an administrator
2. Type brew install docker (mac) or winget install docker.dockerdesktop (windows)
3. Click the appropriate options and wait 10 min to install
4. Restart your computer
5. Run Docker Desktop
6. Accept the license agreement (opens in a new tab)
7. Windows Users - You may need to install an updated WSL2 Kernel (opens in a new tab) along with some minor tweaks and restart again
8. Verify that Docker Desktop shows you the Getting Started Image (opens in a new tab)

Full play-by-play explanation (with pictures!)

So we are going to work with PHP, and we are going to be running some code on servers, which are computers that will not be exactly how yours are set up. How do we make sure that the computers have all of the necessary tools and correct versions of libraries that we use on our computers? Enter docker, which is software that allows us to create and use containers (opens in a new tab).

Containers are like "mini-OS+software" that can run on top of many different pieces of hardware. Containers have all of the libraries/tools/programs that we need to run code and nothing else. It goes beyond the VM route to be much stricter on what it includes. It's a really neat idea that's marred by the onboarding and existing tooling. But think of all the times that you've already experienced the idea of "works on my machine" when moving across different computers. This is exacerbated in server-side web development because we leave the comforts of having browsers that have compliance and show a limited subset of what we want. We are now getting into running any arbitrary code that we want on a server, which makes things much more open-ended. We want to eliminate that.

Docker (opens in a new tab) is the tool that we will be using, which has become the industry standard. Some of you have already worked with it as well.

Option 1 - Via the command line

This is the way you should do this! Remember that we installed winget and brew last time? We don't have to wade around, just open up Terminal (or PowerShell) and type one of the commands we have up above. You may have to run them as an administrator to get this to work.

Windows

winget install docker.dockerdesktop

Mac

brew install docker

Then just sit back, take a walk outside and come back. It's easy, isn't it? So much better than having to download and do all that stuff.

Once you are ready, go ahead and open Docker Desktop.

Additional aside for Windows

If you are on windows, you may see something like this:

You'll need to click on that link (opens in a new tab) to download the package at the top of that webpage.

Run the Windows Subsystem for Linux update and ensure that it completes. Once it does, then click the Finish button to closer the updater and then click to restart your machine.

If you are having trouble or have not done anything regarding WSL, please see this guide on WSL (opens in a new tab)

Option 2 - Via the Web (opens in a new tab)

The one issue with docker is that it is a heavy program. From the link that I provided, you must pick your operating system/architecture.

Once that downloads, install it like you normally would any other installation. It may take a while. Before it starts installation it may give you a few options. For me, I ended up unchecking the desktop shortcut and then I clicked OK.

For my old desktop, it took a good 5-10 minutes for docker to finish and then it greeted me with a restart message.

I highly recommend you restart your computer on this one. Normally I don't and just start working, but unfortunately, Docker is a heavy program that starts up with your machine and does all the stuff you may not like. Feel free to uninstall it at the end of the semester. Once you restart, you'll be greeted by a scary-looking message about their license agreement.

I just clicked accept, since we are an educational institution. This wasn't the case until recently as open-source projects start running out of funding and the like, they are trying to stay afloat. Lucky for us, we get out scot-free here.

You'll know that you are ready to move on because Docker will start up every time your computer starts. You'll notice an icon in your taskbar . You can go into the settings to specify it not to run on startup.

Troubleshooting Docker

Unfortunately, for those of you who are on windows, this is one area where I think Docker performs much more poorly compared to Mac and Linux. So what I'm going to do is to show you a few different things that I ended up noticing and what I did.

An error occurred

One thing that I've seen before is an error occurring like this one below . All I did was click on the Reset to factory defaults. That does make that error go away.

Docker desktop not starting

Something else I noticed that is more sinister is that docker would just not start, you would see a notice like this:

For this, what I did was follow this StackOverflow page (opens in a new tab) about changing the settings.json file (in C:\Users\<your windows username here>\AppData\Roaming\Docker\settings.json) to "wslEngineEnabled": true

You could also try typing in an elevated command prompt.

& 'C:\Program Files\Docker\Docker\DockerCli.exe' -SwitchDaemon

And then pressing enter. It will take perhaps a little bit of time. Then you'll need to restart your computer.

How do I know if I can move on?

If after restarting, you see a welcome screen once Docker desktop starts up that asks you to get started.

Click Skip tutorial. Once you click the skip tutorial, you'll be presented with the dashboard, which will be empty. You can ignore that for now.

One option that you can select if you want is to stop Docker Desktop from starting up when you start your machine. You can do this by going to the settings and unchecking the option to start it up, and then clicking the button Apply & Save.

Phase 2: Using Docker

High-Level Overview for Phase 2

1. Understand the difference between an image and a container
2. Our first container - In Terminal, type docker run -d -p 50127:80 docker/getting-started and ensure that when you go to http://localhost:50127 (opens in a new tab) that you see a webpage
3. Once verified type docker stop $(docker ps -q) to stop the web server from running.
4. Let's finally grab the docker image we will use for development by typing docker pull phpstorm/php-71-apache-xdebug

Play-by-play for Phase 2

Docker Desktop is a bundle of many tools related to managing containers. Because we are learning and utilizing the command line here more, I'll show you some commands related to docker and its use.

Image vs Container

Before doing that, I need to explain a couple of terms that are important for understanding how to work with docker. While the way that I explain this here is not completely accurate, I think it works well to understand the difference for now.

The first is the idea of an image, which can be thought of as a class. The image can be thought of as a replica that has all of the data needed to set up that mini-computer. In contrast, a container is a run-time version of that image like how an object is typically derived from the class. You can create many containers (just like you would make many objects) that are all based on the same image, and we can change the underlying data in each container if we needed to.

Your first container

Now in your terminal, type docker run -d -p 50127:80 docker/getting-started, and notice that docker goes to work. If you read the logs closely, you'll notice that it does not have a local copy of this docker/getting-started image, so it downloads one from Docker's hub. It then will then create a container based on it.

• the -d flag allows docker to continue running in detached state, which means you'll be able to enter more commands in the terminal while the docker container is running Not having this flag will cause the docker image to run in that same terminal and you would not be able to enter any more commands unless you open up a new terminal.
• the -p flag allows us to translate the outer port (50127) (which is what is in your personal computer) with how docker sees the ports inside the container (port 80). For example, this getting-started container is running a web server. You may recall that web servers typically use port 80 to respond to web traffic. What happens though if you already have a web server running on your local computer and you want to run this container that will use the same port? Or what if you wanted to have two containers running at the same time - and both want port 80 to be used? Instead of having that type of conflict, we merely reassign the port without modifying any of the code. That allows the code and libraries to stay the same, and we as the ones running the containers can map the containers to different ports.

Now with the docker image running, please visit http://localhost:50127 (opens in a new tab) to see that you do now have a webserver running, and yet you did not install any webserver software, like apache or Nginx. Nginx is actually what is used in this getting-started image.

Now that the image is running, we'll want to stop it from running and remove both the container and the image as we won't be using this example anymore.

First, run docker stop $(docker ps -q) to stop any running containers. This will stop all containers. If you wanted to instead stop a particular container, you can use docker container list to list all running containers and then type docker stop <start of container id you want to remove> to kill that particular container.

Stop gives the container about 10 seconds to try to clean up and shut itself off. If it does not stop after 10 seconds then docker will send the kill command and the container will be pwned. Docker does a good job of figuring out which container we are referring to once we provide enough of the id to make it different than others (a couple of characters is usually enough)

docker rm $(docker ps -qa) to remove the container (which was like you building a computer with those specs) and then finally docker rmi $(docker images -q) to remove all of the images. You should not run this last command very often, but in this case, we want to remove the image that we downloaded since we won't be using it anymore.

Here's the cheatsheet for other docker commands

Our development container for now

Now that we understand that this has all of the programs we need, let's have it do the heavy lifting finally.

Instead of us installing apache, PHP and XDebug, we are going to pull a docker image from docker's cloud (which it calls docker hub). In Terminal, type:

docker pull phpstorm/php-71-apache-xdebug

This time we are not running the container, but merely downloading the image to our computer to prepare for the next part.

Phase 3: Set up PHPStorm (opens in a new tab)

Before you continue or anytime that you work on this phase, you need to make sure that docker is in fact running to work on your code.

High-Level Overview for Phase 3

1. Install PHPStorm (opens in a new tab) by typing winget install JetBrains.PHPStorm (Windows) or brew install --cask phpstorm (Mac) in Terminal
2. Activate your student license (opens in a new tab)
3. Configure your appropriate keymap
4. Configure docker as the webserver and PHP engine that PHPStorm will use
5. Learn how to commit and push changes

Play-by-Play

PHPStorm (opens in a new tab) is PHP and web development editor from JetBrains (Make sure that you activate an educational license (opens in a new tab) from them). I had mentioned Webstorm previously, PHPStorm is the same thing except it has PHP and SQL support. I will be supporting PHPStorm instead of VSCode for the remainder of this class jumping to full-stack development, as I was not happy with the current state of the extensions for VSCode.

If you haven't already, use the command line to download PHPStorm.

As you are downloading the product, make sure that you fill out the student application (opens in a new tab). While PHPStorm offers a 30-day free trial, we want to start the application process early to avoid any hiccups. Jetbrains provides free access to all Jetbrains products while you're a student.

Once it finishes installing and you pass all the permissions, you can open it up.

You'll be greeted with a simple window

Most of you will choose not to import settings and click OK.

At this point, you'll now move on to the project page, where you'll need a project. You can select the Get from VCS project:

Before moving on, make sure that you end up clicking on this GitHub Fullstack assignment (opens in a new tab) to generate a new repository that we will use for our full-stack development. Remember that to go through all the steps as we outlined in the lab0 process

Once you have a repo in our comp127 org, you can then move on to the next part, where I'll ask you to provide a webpage to login via GitHub. See here:

After waiting for a bit, you'll then be prompted to login to GitHub.

After clicking to login, PHPStorm will launch a webpage; follow the instructions to login and close the page when it says so. If you do this, you should see at least an attempt at trying to download the project. You can also try to use the GitHub option, and you should see your own projects after logging in. However, you may still get that the clone failed.

If you are having trouble just logging in, you can merely go back to lab 0 again to follow the steps on generating a token (opens in a new tab).

The other permission I had to set was to allow JetBrains access to comp127's organizational space. You can do this by visiting GitHub's apps settings (opens in a new tab) and then clicking on JetBrains IDE Integration. Click on that and then you should be able to see the heading Organization Access. Click on the Grant button next to comp127, and now try cloning the project again.

Once you finish this, and the cloning does not fail, you now can get to this step:

Some of the extensions required payments for professional features, so we will switch to PHPStorm to have a modern development experience. Since by this point I have become accustomed to eclipse keyboard shortcuts, PHPStorm provides an ability in its settings to use an Eclipse keymap. They also have other keymaps as well. For me this allows me to not have to worry about learning new keyboard shortcuts, I can continue using handy shortcuts from eclipse like: (Alt-Up/Down, Ctrl-D, Alt-Shift-R, Ctrl-Shift-Up, Alt-Shift-M, or Alt-Shift-H, to name a few).

Configure your settings

When you have a new dev environment, it often is beneficial to spend a little bit of time tweaking things to your liking (but not too much initially, spend no more than 10-15 minutes initially. I'll go over a few different settings that I've found useful.

To switch your keyboard shortcuts, open settings by selecting File->Settings (or do Ctrl-Alt-S) and searching for the Keymap tab in Settings. Pick the file that will work best for you in navigating using keyboard shortcuts.

One setting I would like you to modify is to add some additional inspections to PHPStorm. Having PHPStorm do more of the heavy lifting in looking for errors is one thing that can speed up your development. One in particular we will add is the W3C CSS validator. With the settings window open, select Editor->Inspections in the left hand menu.

FYI: If you do not see the CSS Validator, please install the W3C CSS Plugin from JetBrains. This can be accessed by going to Settings->Plugins->Marketplace and search for the W3C CSS Plugin from there.

Once there you'll see a search box in the right. Type w3 and select the W3C CSS Validator as part of the settings, and make sure the checkbox is selected and press OK.

As for plugins, unless you are heavily invested in another code editor like vim or emacs, then I don't think you need to install any additional plugins.

Connecting Docker to PHPStorm

We'll need to do some additional setup so that PHPStorm knows what version of PHP and web server we will use. To do this, please follow this set of instructions, which is dependent on OS.

Open up settings again, and this time click on Build, Execution, Deployment and then click Docker. You'll then see a plus icon.

Once the plus is clicked, you'll see docker menu options on the left-hand side. For windows, just select Docker for Windows. For those of you on Mac or Linux, you would just select Docker for Mac or Unix socket for Linux.

Click Apply and OK to close the window.

Creating the runtime environment for the PHP library from Docker

Now that PHPStorm knows about the Docker engine, we will let PHPStorm know the PHP version we'll use for the project.

This will be done by using a file that we have already in the repo called docker-compose.yml.

When you open up docker-compose,yml, you see a few different instructions that tell docker how to set up the container. Particularly it is building the image from the Dockerfile (which is just a couple of lines saying to grab an existing image that is hosted and to copy these files to it). On lines 5-6 the yml file tells docker that any requests to 51127 on your computer should be translated to port 80 for the container (remember you are running a small computer). Lines 7-8 link the existing directory (.) to another directory inside the container (/var/www/html which is the directory apache will use to server files), as well as an option for how to use xDebug, a PHP debugger.

Please change the port from 51127 to 50127.

Open settings one more time, and this time we will specify the interpreter that we'll be using. Because our interpreter is in our Docker container, we'll use the version specified in that container. For this to be running for me I had to hit refresh. You may also need to ensure that the docker services are running, which you can do by pressing the play button on the docker-compose.yml file.

Specifying the external web server

One thing that I found frustrating was wanting to get the server up and running to check the changes and seeing these little icons next to the side of the machine . When I first tried to click on them it would take me to a default version of what PHPStorm thought my server would be. Let's change it so that it opens our docker service instead. Here are the steps I followed to get our external Docker server connected.

Once I did this, clicking on those icons while developing would cause me to be able to launch the web browser with the PHP code loaded.

Now our workflow for making changes should be to make an edit in a PHP file, then to either run it via the browser or from PHPStorm.

Running it via PHPStorm will provide you with the raw text that PHP would output (essentially the HTML code plus anything extra that PHP outputs), while reloading the page on your browser would allow you to see how that code translates to a webpage.

For example, let's type this highly sus code in a new file, which you can create called hello.php. To make a new file, right click on your project directory -> New -> PHP File.

Let's set the preferences for your new file like this:

Notice that phpstorm started you off by opening a php tag. We will leave that there. Make sure your file by the end looks like this.

<?php
    print "Hello, COMP 127!";

Yes, the closing part of the PHP tag in a pure PHP file is optional at the end of the file (opens in a new tab). You can see your changes and run your file in one of two ways.

1. To run a file for the first time, right-click in the file contents area and select Run as PHP File, which sets up a default run configuration (you had something similar in eclipse). After that, you can use the keyboard shortcut to rerun the program (for me with my eclipse shortcuts, it's Alt-Shift-X)
2. Your other option is to just click or to visit via your webbrowser, my URL for me was http://localhost:50127/hello.php (opens in a new tab). You can also use one of the browser icons that we mentioned above to launch the web browser with that page loaded. The difference that you have to remember is that your web browser will run the PHP code, but will then try to interpret the output as a webpage. We will cover this in one of our first lectures.

Pushing changes to GitHub

Remember that we'll be committing our changes often. To do this, you can either leave the commit pane open on the left or you can keep clicking Git->Commit, which can also be found sometimes as a checkbox which can be enabled via a toolbar to the right of the run icons. See here:

Once you click the button, it will open up the git side panel that is on the left (which is also clickable).

There are two folders in the commit window:

• Changes
• Unversioned Files

Changes are files that will be tracked by git that git has noticed are different, while Unversioned Files may be files that git is not currently tracking in source control. Instead of a sync changes button like we had in VSCode, to push the changes, you can either:

1. Click in the Commit toolbar In either case, a new push window appears that shows you everything that will be sent to the server. Make sure you click push and you can verify the changes have been uploaded by visiting your repo on http://github.com (opens in a new tab).
2. Click the push icon in the git toolbar. .

Phase 4 - Cloud Hosting

High-Level Overview for Phase 4

1. Create a [fork of your account as we did with lab0.
2. Understanding why we are switching providers
3. Create a mogenius account
4. Connect the repo to your account
5. Create your service by exposing port 80
6. ...
7. Profit?

Play-by-Play for Phase 4

Creating a fork again

Before we get started with our deployment, we will use the same deployment pipeline that we've done previously with lab0, where we'll use a fork of your comp127 repo as the actual connection to the cloud provider. This means that you will need to fork this new repo, as we did before. You'll need to make sure to sync the repo as well anytime you make changes to flow through.

The rationale for moving to another service provider

After creating the fork, we will now connect a hosting solution to our PHP project. Instead of asking a server to merely host files and provide them to a requesting client (which is what we were doing with a static site), now that we have introduced PHP, we need a server to execute commands as well. This can be both risky and expensive for cloud hosting companies! Imagine giving the internet dedicated use of your computer. This makes deploying PHP projects trickier. Luckily the deploy process by using docker is actually simpler when it comes to the web. Because we'll be able to deploy our docker container as it is, there is not much else needed and we know that will work since all the files will end up being in the container.

Most cloud hosting platforms (Google cloud, AWS, Azure, IBM) offer some sort of credits for you to get started with working with their hosting platform. This has always left a bad taste in my mouth and very hesitant to try other hosting solutions (not to mention to have you all try those as well) Some have even partnered with Github (like Heroku) to give you free monthly credits or some extended trial period. However, my experience with AWS in particular when I tried it out a few years ago was less than ideal. I made the mistake of logging out of a console before stopping a service, and suddenly got a surprise bill at the end of the month. I would not like any of you to get charged like I did for making what I thought was a simple mistake. This has made me search far and wide to look for a provider that would:

1. Host docker containers
2. Be free (without a credit card signup)

The options are slim, but I did find one, and that site is mogenius (opens in a new tab)

If you prefer not to use it and would rather try using AWS, Google Cloud or other industry standard providers, then please do so! Really the whole reason for step 4 is to get your project online.

Realize that we're not responsible for any extraneous credit card charges. I have heard that things have gotten much better and you can even set alarms, but again, after my experience, I was not happy so I have avoided most major cloud platforms. Instead, I have looked for smaller hosting providers that still give us the options we are looking for as we don't need anything that is blazing fast. Before this semester, that provider was Heroku. Unfortunately, Heroku just recently made its pivot away from providing free cloud services. While Heroku just recently announced that they will be supporting students through the student developer pack, they still require a credit card "for verification purposes".

Getting started with mogenius

This tutorial though will be for us to use mogenius (opens in a new tab).

PLEASE HOLD OFF ON CONTINUING HERE UNTIL FURTHER NOTICE

Start off by going to their site and signing up for an account (opens in a new tab). You'll need to verify your email address with them. You can provide any email address that you have access to.

Once you click on their email to verify and input the code, you'll be able to start.

You'll be greeted with a page that looks like the following image:

Connecting the repo to our account

Click the Create cloudspace button at the bottom of this page, and provide your cloudspace with an internal name: any name that is just for your reference. I named mine Server Side Lab.

Keep the selected plan as Free and then click Create now.

Now that will launch you to our next page which looks like this:

We will select the option Deploy a repository with a Dockerfile. Note - If you somehow no longer see this deploy wizard, You'll need to click the add button in the stages submenu, which can be located from the dashboard (I also switched to dark-mode), and select the Bring your own code option.

That should bring you to a page that looks like this:

1. Click the Continue with Github button
2. Click the button on the GitHub page.
3. Select to install mogenius on your personal account.
4. Click the button.

Once authorized, you'll be back on the same page as what we have here shown above, except you'll see your github picture up top (if you have one) as well as options to select a repo below:

Go ahead and select the full stack lab git repository. If you just recently had the repo connected, you may need to Refresh the web page.

Create your service and expose port 80

Once selected, you'll notice the Service name as well as a few other options being automatically selected for you, including the git branch, dockerfile and context.

We will leave all of these options as is, except for the last one, which will be to expose port 80 (which is our internal server port to the webserver) and then click Create Service.

We want to do this on mogenius so that regular users can type a regular URL (without needing the port) and have it default to port 80. Why didn't we do this on our own machine? We wanted to map port 50127 to port 80 on our own machine because we may have other services (like a web browser) running on our own port. Similarly, if you don't have firewall protection on your computer, you could theoretically run the web server from your own machine and folks could type in the IP address of your machine to connect to your computer, so we took that extra precaution. With mogenius, they are setup to be a secure web service, so we expose port 80 there.

After clicking on it, you'll see that there will be multiple processes running here as it will take some time to get the deployment started.

It took a few minutes, but afterwards, I clicked on the hide next time checkbox and closed the process window. You'll then be back at this dashboard, where you'll click hostname and then you'll see external hostname. That will be the URL for your web server.

Uhhh

Your url will have some crazy subdomain followed by mogenis.io, along with a copy button, so click the copy button and enter in the URL. You can click the URL here, but you'll get some rule that says Forbidden. 😓😓😓

But wait! Notice a couple of things... First, we can tell that this URL is running Apache, which was the web server we started with. Also, this may remind you of what happened with our cloudflare lab as well. We don't have an index.html here, we have a hello.php file, so once you attach that at the end of the URL, success!

Profit

That was it for now.

A couple of things to know. If you aren't running docker desktop, you can (and probaly should) shut it down. You can right-click on where your services are and say Quit docker desktop. When you need it, you can place it back up and running again. You can also see your resource allocation in mogenius for how much of different resources you are taking for your free tier. You may need to stop and start services so that things are working and within the limits of what's allowed.

You can probably do the same with the service that we provided at some point

This was long and mighty lab (at least to write), but I hope it helps to further your understanding of the types of workflows we want to utilize. If you have any suggestions, please let me know, but also please make sure that you ask for help and help out your peers in discord as well!

Now with this ready, we can start talking about PHP more in the coming lectures.