[Help] Stepping up my back end workflow skills: url-shortener as example

Hello coders!

I am pretty new to back end and so far but I have the feeling that I am approaching this project tin the wrong way.
I am curious especially about the “ideal” workflow and how a project is supposed to look, behave and interact with the user.

I am gonna take my url-shortener as an example for this.

In creating my project(s) so far all I did was creating

*server.js file that handles the requests and responses.
*index.html that contains a plain basic html with the program explanation.

But when I look at other repos on Github, I can see that many of my favorite libraries and project contains multiple folder, with multiple .js files.

Another big issue I had with this project was hiding my username and password from the public Github repo.
My workaround has been creating a config.js file that has been added on the .gitignore; but then creating a different branch on github where this file is not hidden for the actual deployment on Heroku.

Are there better practice / techniques to deal with this?

So I am curious about the best practice and ideal workflow when dealing with back-end in order to write a good readable code accessible to all the interested peoples.

Here the Github Repo.
Here the live app.

p.s. I am aware of the existence of this wiki articles on the forum; I am just curious about you real life experiences.

<3 Thanks all and as always happy coding <3

1 Like

I’m not expert enough to answer this with any real authority, but these are the same issues I was dealing with around the time I made my URL shortener, too. In fact, after someone on Gitter told me mine was a mess, I started again from scratch and completely redid it!

There are two main parts to your post, so I’ll try and address them both.

Firstly, the easy one: Hiding your private info when you push a repo to GitHub.

When developing locally it is common to either set environment variables for your computer, or to use a Node package called dotenv, which basically emulates the same functionality. These methods allow you to access special variables using process.env.[VARIABLE_NAME] (eg, process.env.MONGO_URL, might be one that you use). That variable is hardcoded into your system or dotenv file (which can be .gitignored) so that the code you push is sufficiently hidden.

dotenv can’t be used on Heroku though, because it is .gitignored, but you can set environment variables on heroku either from the terminal: heroku config:set MONGO_URL=mongodb://username:password@blahblahblah, or you can set them using the Heroku dashboard under your app settings. They call them config variables.

Your other question about project architecture is open to debate and interpretation. It doesn’t hurt to read around about that stuff.

My takeaways so far have been that for tiny ‘Hello Worldy’ projects, architecture doesn’t matter. Once you start having a couple of hundred lines of code though, it gets a bit more manageable if it is organised.

How you organise it is up to you, and as long as it is consistent and sensible then it’s ok.

A common configuration for me in my backend apps has been like this:

  • one file called server.js which handles my express and mongo set up, including all the appropriate middleware and making the appropriate connections

  • routes.js handles the stubs of all my http requests. I don’t put all the things each route handles in this file, literally just the URL endpoint and a named function squirrelled away in another module. eg router.get( '/', userQueries.getPolls );

  • controllers/userQueries.js contains the functions that the routes file depends on to do DB operations. One of the functions in there might look like this:

Expand
getPolls: function( req, res ) {
    var db = req.db;
    var polls = db.collection( 'polls' );
    polls.find().toArray( function( err, docs ) {
      if (err) {
        console.log( err );
      } else {
        res.render( 'pages/home', { docs: docs, user: req.user } );
      }
    });
  },

  • views/pages/home.ejs (and a bunch of other ejs files) are my view templates. I also have views/partials/header.ejs and some others for page snippets that get repeated (nav bars, CSS/JS links etc)

And…I think that’s my basic structure at the moment.

This is not perfect, and may not be entirely best practice, but the reason it benefits me is that if I suspect a bug is being introduced because one of my routes is a bit wonky, I can look in a 50 line file dedicated to routes, not an 800 line file that handles routes, views, database connections, database queries and anything else!

If you want to see all of that together, you can see one of mine here: Voting App

1 Like

Modularise as much as you can. Once you start with the larger back end projects, having separate independent modules handling your responses, views, database etc helps a lot. This way, if something breaks, you can remove or fix it easily and your code won’t be coupled tightly.

I have a similar structure to @JacksonBates 's,

  • I separate all my route handling into a express.Router and keep them in a folder called routes.
  • My passport.js configuration is in /config.
  • Then I have a folder for static assets, which is split into dev and dist. The first one has SCSS and Babel files which compile to dist, and are uglified.
  • Then there’s a folder for views, which has pug files
  • Finally a folder called mongo, which has Mongoose schema for my database

I use gulp to automatically compile static files then refresh the browser on save, and to restart the server when server logic files are changed, and then automatically refresh the browser. I’d definitely recommend to get used to gulp, because it’s so much faster to work with it. If you know webpack though, there’s no need to use it.

Disclaimer: I’m not a pro, and this is just what I used for my first and only (so far) dynamic back end app: here’s the repo if you’re interested.

Thanks for the explanations @JacksonBates and @imtoobose. I’m starting my first web app project and am like a deer in headlights right now trying to figure out where to start.

I’ve done some reading and am going to try to set up an MVC model, which sounds like it’s similar to what you guys put together.

Moving from the API projects to the Web App projects seems like a huge leap…

Thanks @JacksonBates and @imtoobose for taking time to answer :slight_smile:

As @jer244 said, I will try to improve my MVC model using your suggestions and ideas :thumbsup:

I’ve done some additional reading on MVC trying to wrap my brain around it… I get it in theory, but have been having some trouble implementing it…

I came across this answer to a Quora that really helped. Tasos Soukoulis put it pretty simply by basically saying that your server should create an API that your front end should consume, and to communicate between the two using JSON.

Maybe it’s because I’m just coming off the API projects, but thinking about it this way really made sense to me. I don’t know if it helps you, but I thought it was a great way to think about it.

I also came across a great blog post that covers some security issues that are possible when you use body-parser. It’s a pretty short read, and I thought well worth the time.

Hope this helps.

1 Like