Transcript
Ferguson: I didn't invent JAMstack, that was a mistake. Though it's funny because it's like part of my name, like JAMstack, Jamund. Yes, this talk is all about bringing JAMstack to the enterprise. I guess the question that I want to start off getting people thinking is how many of you think that your apps are fast and stable enough as they are right now? What stack are you using, man?
Participant 1: React.
Ferguson: React, ok. Tell you a little bit about myself, I've been at PayPal for a couple of years. I came in during this big thing, we're moving from Java to Node, that was the future five, six, seven years ago, and I had already some experience working in startups in that area, so I brought in a lot of expertise there and tried to get us really good at making apps that were stable, easy to develop, working fairly fast. I became fairly good at this, I would internally help teams work on these processes and work on their frameworks and stuff to make sure that it was going well. I worked primarily on the send money team. How many of you have ever sent money before with PayPal? You know the tools that we're working on, you know these are used by a ton of people, which means we're probably one of the bigger Node shops out there.
We have a lot of big problems and we got good at solving these problems. Last year, I was speaking at QCon about something called building scalable and dependable Node.js applications. I thought of myself as being quite good because I've had so many problems that I learned how to fix, that I finally was ready to share those with the world. We talked about type systems and talked about all kinds of other tools, linting, debugging tools, static analysis, air handling, anything that we could to build stable applications.
Even though I'm out there presenting, as the expert, telling people like this is how to build stable Node apps, the reality was when we got home, we had a lot of problems with our Node apps at PayPal, weird. Anyone giving a talk here today about something, when they go back to work, they still have all the same problems they're telling you they know how to fix. It's just reality.
Three problems, in particular, one was this lack of stability, I'll go into a little bit. Two, is these incredibly slow deploy times because we have these massive application server pools. Three, is this client-side performance, mostly because our bundle sizes were so big. Hopefully, that my presentation today will address some of the issues you've run into. Hopefully, you'll start to get solutions and even if the exact thing we ended up with isn't perfect for your teams or your projects, maybe you'll get some ideas that you can apply there.
Stability
Let's talk about stability. Every time we have a crash, we get a crash email, at PayPal. It's very nice, “Hello, your server crashed” and I am ridiculous so I subscribed to every team's crash emails because I wanted to see what was going on. At some point, last year, I got so many crash emails that my outlook crashed, which you're allowed to laugh, it's pretty ridiculous. We had a bit of a stability problem. It wasn't just one team, every team had a little problem here and there and that adds up to a lot of crashing. The other thing that I mentioned about the deploys was we had a lot of servers. Though we have a lot of data centers, and availability zones, and maybe this is sort of a PayPal scale problem because we have things in all sorts of countries and all sorts of places. We really want to make sure stuff doesn't break, we have this stability checking all through the deploy process. Deploys can take hours. The send money at the PayPal that I worked on, we have something like 400 Node servers, and realistically, they're all taking like a little bit of traffic, but we really want to be sure that we can scale if we need to. The easiest way to do that is just throw lots of servers at the problem.
This slow deploy process can be really demoralizing for developers, if I find out about a bug, and I fix a bug, and it's Friday, or Thursday night, then Friday morning, "Ok, I'm going to get this ready for production. If it's going to be four hours to deploy, I'm probably just going to wait until Monday." A lot of things end up stacking up when deploys are slow.
Performance
Next thing that we're dealing with, I do want to mention is performance, a client-side performance, especially. If you click through all the different pages, summary, activity, send money, wallet, these are the main pages in the PayPal application. These are all different apps, served by different Node servers, being developed by different teams with different JavaScript bundles being served. If you go look at the client side performance, you'll notice a pretty big gap between like content loaded and load. The time it takes for all that JavaScript and images and stuff to load is pretty big.
If you sort the size, you're going to notice in some apps, like the one I worked on, we have this chunk bundle with 1.3 megabytes of JavaScript that has to be processed. The older the device, the slower it's going to be, it's not great. Tools like webpack and stuff made it really easy to bring in additional tools and made it really fast to develop exciting applications. On the flip side, it gets easy to build these giant bundles. Some of the other apps we've got here, a meg or two megs of JavaScript on the activity page as well as another half a meg almost of CSS that has to get loaded and processed, just to show a couple lines up there. It's a bit of a problem.
I think training is key on this stuff, I could go around and other experts could go around. We started doing this last year, I set up a team called the consumer web platform team. We had a performance expert, I call it Griffith and she would go around, start planning with each individual team, "Ok, how can we improve your performance." I would just sort of going around trying to give teams feedback on how to make their Node apps more stable, but ultimately, this process, is not great.
We feel like if doing the right thing with your app is too hard, if it becomes too difficult, and you have to manually teach every single team how to do things the right way, maybe it's time to just rethink our architecture. Maybe the problem isn't the people, people are great, they're trying their very best. If the tools make it too easy to screw up, maybe we just need to use some different tools.
I don't know if you felt like this, but last year I came to QCon. I got all this technology envy and literally, for two months after I came to QCon last year talking about Node, I tried to convince everyone in PayPal to use Go because it's supposed to be fast and cool and stable and whatever. Ultimately, it's like apparently retraining 1,000 JavaScript developers to a new language is hard so we decided not to do that.
Cloud Functions, there's going to be a great talk, immediately after this about Cloud Functions, I think these are really powerful and solve a lot of the stability problems that I'm having. When we looked into them a little bit, we have some security questions and some performance questions that we weren't able to address in the near term. We're not able to use them now, I think long term, they'll make a lot more sense. In particular for us, because we store a lot of encrypted data, we were keeping that inside our private cloud. Even if we're doing some stuff in a public cloud, making those connections from public to private cloud has some costs with it and doing that a lot can be really slow. There are some challenges there.
Of course, other technologies, like machine learning seem really cool, but they don't necessarily apply to the types of problems that we're having here. Despite all my excitement about new technologies, and wanting to try all these new stuff, realistically, I was stuck with our current stack in ways that we could just build on top of what we already have. This is what the average PayPal stack, or the average stack for front end apps looks like. PayPal, we got back end services. We got back end services, not really, working and Java and C++. Then our application servers and sort of see that like black background, anything in that black background is what I need to worry about as like a product engineer, a UI engineer at PayPal. I've got my Node app that has a renderer, in this case, like JSX, we also do still some dust rendering, and then maybe a REST API or GraphQL API in the same app that generates HTML, which also pulls down JavaScript and CSS living on a CDN somewhere. Does this basically look like some of the stacks you're working on? I think they should be fairly familiar to most people. Because this wasn't quite working as well as I wanted it to, we are having enough problems with it, I started seeking for some inspiration about how we can improve it.
Backend Services
One quote that came up that I really liked was Nicholas Zakas here, talking about isolation. Here, he's talking about lambda functions, but you can apply these principles in many different cases. Can you have one component that can fail without bringing down other components? How can we apply that to our infrastructure? Well, we have a problem sometimes with our infrastructure that our renderer will break. We support a lot of countries, a lot of different languages and locales, and so sometimes, if we don't have enough, maybe some files didn't get generated properly for one language, when someone from that country goes to use the site, maybe their experience will crash and that's going to bring down the API, because it's all just one giant process.
One thing that we could do is split those up, you have a renderer, and you have your API, and they don't break each other. Now, if something goes wrong, like the other thing can stay up, so it's happy. This will help us inch towards stability. It's not quite enough, though.
The next thing that we dealt a lot with was the REST APIs the individual teams are building were failing, and usually it was because when we talked to the backend services, which are REST servers, we thought that they would be returning data in a particular format and a particular shape, but somehow that third or fourth property down that object response, that object.data.whatever is not really existing. I know you all have dealt with the exact same thing, you expect this property somewhere down the chain to exist, and it doesn't exist.
This happened enough that we really struggled with a couple different solutions and there're obviously a lot. One thing architecturally that we were able to do is put in place a GraphQL layer in between our apps and our services. Not all teams are using this, but where we experimented with it, it seemed to work pretty well, because what we're able to do now with GraphQL is say, "I want you to send me these four fields," and we have a high amount of confidence that those words fields will actually exist in our application. If not, if there's an error, we have a pretty clear idea of what the error is going to look like. With GraphQL, you're going to have a greater confidence in your responses, you're going to have a little bit more organization, especially if you're doing microservice world, which is very popular, I think, when you stick it all through a GraphQL schemas, you normalize those microservices into a single way to communicate with them, it works pretty well.
Then, once we brought in this, something became immediately clear, which is that we didn't actually need to have a separate REST API for each and every app. If we already have all the data that we need exposed through GraphQL, we can just have a thin proxy that adds some authentication, and then our front end clients could access GraphQL directly, and potentially, just remove that whole part of our application stack, which is awesome. Here's the thing, the best way to avoid bugs in an application stack, is just remove a whole part of it, cut that piece off, it's no longer causing any trouble for us.
Then I got really excited, if removing something makes the system more stable, what else can you remove? That's when things got possibly out of hand. What I realized, and I wasn't the first person to realize this, but what you can do, actually, instead of having this render server spitting out HTML, you can actually pre-render the HTML in a build step, and stick it over there on the CDN, with all the other static assets. It's just HTML, we know that's what's coming down to the client. Why do we have to generate that at runtime, why not a build time?
Then we can delete that entire server and all the services that was depending on and sort of massively simplify our stack, so that now we just have something like this. I don't have to worry about, as a front end developer, I can just build my UI, talk to the shared GraphQL layer that like some other team has to worry about, I don't have to maintain that stuff. It just has all the different schema, the GraphQL schemas I'd want. I can just build this thin piece and still do all the things that I need to do to make my app.
JAMstack
This architecture, which I think we fell into thinking about stability, thinking about performance, turns out it's already has a name, it's called the JAMstack. Some of you have heard about this before, let me give you a little bit of background on JAMstack. JAMstack, since our JavaScript API is in Markup, which is really a bad acronym because every website has JavaScript APIs and Markup. It's not particularly clever or unique, but it was invented by the folks at Netlifly, who run this content delivery network, and it works something like this.
When a request comes in to the browser, I will call directly to our content delivery network, so not to an application server, but to the CDN and get pre-rendered pre-built HTML, and bring that back down to the customer, that part happens really fast. Then I can go to the CDN for JavaScript, additional assets and whatnot. If I need to do dynamic stuff, I can call API's in the cloud to do that. That's how the JAMstack works.
Let me show you another example, here's my typical application server running Node. I've got customers all over the world and when those customers want to interact with our application, they can call that application server out in the cloud. That application server has to do some work, it's not just immediately sending HTML, it's got to talk to a service, it might have to talk to a database, if there's enough flow, the database might be sort of on fire, these things happen. Once it does all its work and processing, it creates those packets, and trickles them on down to our customers. That's standard, how the internet works at a super high-level picture.
Let's look at how JAMstack works. I sell my customers all over the world, when the customer goes to request content from me, it's already by them. It's somewhere in a content delivery network close to them. That request happens very quickly, which is nice, because people want things to be fast, they don't want it to be slow, this is so basic, but honestly, it's amazing. If I can just give them what they want without having to go out to the cloud to talk to a bunch of things, why not just do that? I'm going to seem silly not to.
The principle there is you want to keep that content close to the customer. Ideally, we're going to cache stuff on their device, like service work or local storage, just put all the stuff on their phone or on the computer, and then, if it's not there, go to the content delivery network, hopefully, in their neighborhood down the street, somewhere close to them. We heard some talks as well here about edge computing. Maybe we can put some computing on the edge as well as static assets, that's coming up soon as well. Then worst case scenario, put out there in the cloud, where they have to go 500 milliseconds away, it sounds crazy, but let's keep it as fast as possible.
Another principle that JAMstack is few moving parts. In between the call, the request from the client to the server to get data, there's no database, there's no app server, there's no programming language, in between me and the content that I need. It's just already ready, it's ready to go. It's just raw files. Finally, with the JAMstack approach, you can deploy in seconds, because again, it's just files, you can use tools like React and Gatsby, as well as other tools, optimized for view or the other frameworks and you have the single stack.
I've been a full stack developer for six or eight years and it’s been a source of pride. I can debug gnarly, Node.js issues that Matteo [Collina] was talking about earlier. I know how hooks work, I have enough of spectrum that I can talk fluently about the React side as well as deep in the Node side.
Let me tell you where having full-stack knowledge isn't very fun. I have a full stack with my application of bugs that I might have to deal with. The bugs could be on this part of the server, they could be on this part of the server. As much as possible, I would like to shrink the surface area of possible bugs as much as possible, I don't want a full sack of bugs, a full plate of bugs, let me just have a half plate of bugs. Simplifying the model for developers where you're only having to worry about this much of the problem space and that way, if there's a bug, I know it's not in the Node app, because I deleted that part of the application that doesn't exist anymore, so it's probably going to be in the front end layer.
This, of course, depends on the service pieces being really solid, but if you can simplify the amount of stuff you have to worry about, going back to a single stack model is really nice. It also means that I can actually get good enough that I learn how hooks work and figure out all the latest drama in my front end framework community instead of just ignoring it like I usually do. Single stack isn't a bad thing. I think it's something that needs to be reclaimed a little bit.
Hopefully, I have outlined some of the rough benefits of this JAMstack thing, even if it's a bit hazy. I'm now going to talk now in more detail about where it makes sense, what it's good for, when should you use this approach?
What Is JAMstack Great For?
JAMstack is particularly good for static sites, and this is where we hear about it. For blogs, for documentation pages, "Hey, I put it on my blog from medium to Gatsby.” You hear this all the time, this is the new trendy thing. Even marketing pages, with lots of images, pictures, these all work fine in a static site model. By static site, what I mean is you build this application with JavaScript, with the normal things you're used to, and then you run it through a tool that just compiles it into raw HTML. Then you stick that HTML on your CDN and you have incredibly fast user experiences. There can be some interactivity, but it's pretty limited. The JAMstack thing is awesome for static sites. How many of you primarily builds static sites? Maybe someone works on a blog sometimes, most of us are building dynamic websites, interactive websites, like a PayPal, we have a marketing pages, and even those do some dynamic stuff. Really, we have to build interactive web apps, we have to do it.
I wanted to try to figure out how to make the JAMstack thing work. Even though all the talk is about static sites, I wanted it to work for fully interactive web apps. There wasn't fully a name of that, so we started calling these things static apps. Maybe this is a term people know, maybe it's not, but that's what we call it locally, we call this whole thing our static app project. It's fully interactive, real apps with multiple single page applications, but powered completely by static assets. There may be API's that you hit on the front end, but there's application server, like powering these. Just static assets that create these really dynamic and interactive web apps. That's what we're calling static apps, I'm going to show you how they work. Before I described static sites as just basically generating raw HTML, so those just load immediately off the CDN, static apps are a little bit more complicated. If you want to use JAMstack for anything more interesting, you're going to have to figure this part out.
Here's how a static app might work. First, you're going to load the static part of the CDN. The cool thing is, you can get something to the user really quickly. In this case, 300 milliseconds, you can get like a header or a footer and maybe a loading spinner, sort of the shell of the page really fast. Then, once you have that in there, and that'll start loading additional JavaScript, whatever you need, then you can load in the dynamic bit. Once the JavaScript's loaded, you're going to hit a server and say, "Ok, I need to fill out the rest of this page." In this particular case, this is like a PayPal.me profile. I have to pull down the user's image and their name and these things that requires an API call to get the rest of the data. You're probably thinking, "Yes, but server-side rendering, this is making two calls, one, get the page, two, fill in the rest of the data, it's got to be slower." The fact is, it's not necessarily going to be slower. With server-side rendering JavaScript, you're going to hit your edge at some point. This is PayPal's model, if you have a less complicated stack, you don't have to go through so many hops. You hit the load balancer, then you maybe have engine X or something in front of your particular application servers.
Node.js then is calling different services or databases, they're doing their work, then Node.js has to process that into HTML, which Node.js is not actually the best at converting data into HTML, it's a thing that can do but it's not amazing. That goes back through your pipeline. For a complicated enough thing on a slow enough network, maybe two seconds. With the gem your initial load comes in, in 300 milliseconds, so at least you have something in front of the customers extremely quickly. Then if the next thing takes another second or 1500 seconds, it's still going to be on par with the original.
Let me show you a real example, here is the current PayPal.me site loaded on paypal.me/jamund. You can go there to send a person money, this is on live right now. I ran with throttling a basic performance test, it's pretty basic. I set it to say, I have a slow connection, I'm slower CPU. What I found was, the first three seconds after making my request, I just had a white screen there. It just was nothing because it's waiting on the server, it's processing the HTML, it's just a slow thing. That initial page load takes a while when you have a slower connection or a slower CPU. It's mostly server-side rendered, and then some stuff gets loaded dynamically to JavaScript.
Then for about 1000 milliseconds, JavaScript's loading, images are loading, basically, to get the whole page loaded, right now our current version, it's about four seconds with PayPal.me. This isn't terrible, it's fine, it's reasonably fast, it's more optimized than some of the pages we have at PayPal. Realistically, you can just immediately tell that three seconds of white screen isn't great, nobody's loving that.
Compare that with a static app version, using some of the approaches we just talked about, the initial page load comes in under one second. We've been developing a prototype of PayPal.me using the static app approach. I'll go into a little bit more depth about how we're doing that, but this was running in our same production servers that the PayPal.me normal app is running in. Pretty similar type of setup to try and get a fair comparison. This is on our production servers, it's not taking public traffic yet, we're going to do more of this performance testing before we go live with anything. Our initial results look really good. We get something to the user after one second, even if it has just a loading spinner and a header and footer, some of the customers don't even care about the page, they just want to click something in that header, so it's ok. Then we have a second and half of parsing the JavaScript and waiting on that additional content, I put a request, I need to get the name and the picture and all that cool stuff.
About 500 milliseconds after we receive the content, we paint it all to the screen, and for a total of about three seconds to load everything. Our first attempt at creating a static app, our first attempt and putting in production, we're seeing almost a second faster response than a traditional server-rendered application, or at least partially server-rendered React application. This is throttled using Chrome's dev tools, we're going to run this stuff, it's real user traffic over the next couple of weeks and get maybe more precise data. This was our initial read to convince my bosses that I wasn't wasting their time. When we got this number, it was, "Hey, look, we might actually be onto something here." Luckily, this literally just came last week, I was really happy to see that. Yes, it looks like static apps really can work.
Building Static Apps
Let me talk a little bit in more depth about this stuff. Let's talk about how we build these static apps, I've talked theoretically how you build them, I want to get very specific. First of all, you got to think about files, and not routes, this is a one head shift for me. Any route that you have for your site, ultimately, that has to map in the static app or to a specific HTML file living on the CDN or living somewhere in a cloud store. Think about files.
Second thing pre-render as much as you can, just get something in front of the customer that's useful to them soon as you possibly can. When you're talking about something like 300 milliseconds to get that first content on the page, that's great. The customer is going to be really happy, give them something useful. Then you got to think about how often do you pre-render this? You're going to pre-render the page, how often do you want to update that? Something to think about. What APIs you're going to call to low that? Then how can we cache this?
Let's look at an example. PayPal.me settings page? About the URL thing, PayPal.me/settings, well, that'll map obviously to a file called settings at HTML, nothing very complicated there. How much of this page can be static? I grayed out everything that's dynamic, we basically have 80% of the page static. Most of the stuff or just most of these pages just links to other pages to go into more depth. This page actually is mostly usable, after that initial 300 milliseconds, which is pretty cool. If they do need to do something specific with the user information, we'll load that up with an API, and pop all that in there. Then we can think about, how do we optimize this?
To optimize this page, one thing that we found, that I thought was pretty clever, was when a user logs in, we often reuse a lot of this content, their thumbnail image is going to be somewhere, their name, we're going to use the URL they're going to use. We found out that when the user logs in, we can just pre-load all their settings data, all the common settings data for every single logged in page. With that, the page rendering goes from 1500 milliseconds, because we got to call out for this data to 500 milliseconds. We generate the HTML, we already have in cache all their data, we can display it to the page. This is where the JAMstack, I think, helps unlock these exceptional levels of performance that aren't necessarily possible with a standard server-rendered React application. If you can do things like cache important data on the device, all of a sudden you're having like massive performance wins.
Let's talk about a page that isn't so easy to optimize, that's not fun. This is the PayPal.me profile page, which is very dynamic. First problem we run into with this page, and already starting to be annoying, is the URLs. Every URL it's like paypal.me/slug name, like Jamund or whatever your name is. Normally, you'd want to map that to a specific page like jamund.html, and then I could pre-render the whole thing just for me and it loads really quickly, but we have apparently, a few customers at PayPal, a few people. How many of you have a PayPal.me account? Half of you at least. The problem is, I don't know the number, half a million, 2 million, 100, it's enough that I don't want to generate HTML for every single one of you, no offense, you're great people, but I don't want to spend the resources to build HTML just for you. What that means is we have to map, we have to do some kind of mapping, so profile/star, PayPal.me/star is going to end up on a profile.html page. This means we lose a lot of benefits of being able to statically build your page because I'm not building a page for you. There's too many of you out there, unfortunately.
What we get instead is this page that's basically useless on start, we have header and footer and the next button. The input box even is the dollar amount there, it's U.S. dollars, you're probably already thinking, “I don't live in the United States, that's not even relevant for me.” Luckily, we are creating localized versions of each of the pages, too, so that isn't necessarily something that has to be loaded dynamically, I can pre-render a different HTML per locale. Still, it's a lot of stuff, over half the page has to be loaded with an API, which is too bad. We can load it and once it's loaded, we can think about how to optimize it. Optimizing this page is harder, unfortunately, this is too bad, because this is the main entry to PayPal.me that people use, we can cache the user's info. Oftentimes if you go to someone's page to send them money, like a friend, you might go there again. Once you go there once, maybe we could cache that locally for you.
Another thing that we could do is load certain things directly from memory, the URLs already sitting up there in the status bar and the window. We don't actually have to make a call to ask what the URL is, you typed it already. We have that. Then for the image, there's some weird stuff we can do, instead of calling an API to say, what's this user's thumbnail, we could create an image service and say /images/username. That could generate the image so that we can at least start loading the image as soon as the page loads. These are some optimization techniques, maybe advanced for this page, we haven't even implemented them in our version.
I wanted to prove to you this point, which is that even in worst case scenarios, this is the same page that I showed you that test was about a second faster than our current version. The static app approach is very fast, this was an annoying scenario, we don't have a lot of caches, we don't have a lot of things and it's still fast. Think about how much faster it could be in a page that's mostly static. Hopefully, I've convinced you that this is a super cool approach to building apps and something worth considering. Now you're asking yourself, how can I bring this to the enterprise? How can I bring it to my company? How can I convince my bosses, my teammates, my colleagues, that this is something to do because it looks really awesome?
Adoption and Buy-In
Let's talk about adoption and buy-in, two principles here. First principle, you need to solve real problems, my bosses care, particularly when I set up this web platform team and started working on this, they cared mostly that I could improve performance. They wanted to know that we can make our site faster because they figured if we made our site faster, more people could complete their transactions, more people would sign up. From the corporate perspective, my boss's bosses, and all those people, they wanted to make sure I took care of performance. I had to make sure I kept in mind, every time I was presenting JAMstack or static apps, "Hey, look at the performance wins we're going to get here." We have people on our team that are going to go around and team with each team to help them improve their performance. What if we could just improve performance by default for everyone. We're solving real problems, solving the problems that the people care about.
The second thing that was really crucial, was getting this buy-in from other engineers, getting other engineers, my peers excited about what we're doing. One of the ways we did that was by holding workshops. We set up on a Thursday, this workshop in a room that fit 25 people, we got 35 people showing up, it's overcrowded. They came there, we got one of the Gatsby or two folks on the Gatsby team to come and did a workshop and using Gatsby in some of the tools we are using to build static apps.
Just having a workshop like this is so great, it just built up. I literally didn't prepare anything, I just went on the Gatsby page, went to the tutorial and just walked through it with the people. People left thinking, "Wow, I would love to do this at work every day. I would love to use this platform works really well for me." Getting that hype among your peers means that I'm not the only one now applying pressure to use this new system. Now you have other people helping me push on everyone's saying, "Hey, maybe the Node application server paradigm we've been using for the last five years, which served us very well. Maybe we should try this other thing now." Convincing the corporate people by thinking about their needs, then getting the hype cycle going with other engineers works nice. It's a two-pronged approach.
I mentioned we're building PayPal.me. We're also building up tools for people to build these static apps internally. That means we need a tool to generate our static assets and we need a place to host them, which you'll find is actually the hard part of our story. For a tool to generate them, there's this thing called Gatsby.
Gatsby
I've really been super impressed with the Gatsby community and the Gatsby tool itself. You've probably heard of static site generators and Gatsby's lumped in as a static site generator, but it does way more than generating a static site. It doesn't just generate HTML, as you can see in this very complicated diagram from the Gatsby web page, it pulls in content from whatever sources you have, whether it's services, things like that, it then takes all the React components that you're using to build your site and sort of spits out both HTML as well as optimize JavaScript. Then when your web page actually loads, it will hydrate your application. Meaning it will become a fully interactive web application. Some people will describe Gatsby as a progressive web app generator, a PWA generator. I think that's a better name than static site generator because these are fully interactive web apps.
If you want to start using Gatsby, I'm just going to go through a little bit here, you just type Gatsby new QCon or whatever you want to call your project, then Gatsby develop. The Gatsby new will generate an application for you, a starter. Then Gatsby develop is going to actually create a local application development server that you can use to build off of, then you'll have this thing that you can pull up, you can click through, this is all generated for you. The coolest thing, my favorite piece about Gatsby, I know it's a bit of a trick but it is really sweet is, if you put in a new URL that doesn't exist yet, it will tell you "Hey, this page is missing. Go create it right here." Gatsby has a built-in router, you can use your own router and bring your own pieces in. All you have to do is go to the pages folder, create a new file, called, in this case, qcon.js, put in like a basic stub of a page in there. Then all the sudden it exists already, it's already loaded in the browser by the time I hit save, because of all this hot reloading and magic thing, it just feels like magic to me. I love it so much, it's really fun to use, it's an incredible tool. There are definitely other tools you can use for building JAMstack sites, but so far, my heart is with Gatsby, it's really great.
Porting an existing React site, if you're already using React, you just use the same components that you're already using and then Gatsby will just generate a static app out of it. It's not very hard work, which is cool. There'll be a few things you have to work on, but it's not super bad. For example, the PayPal.me marketing page, we grab those components, stick them in our Gatsby app. We had to work out a few things that were previously server-side rendered and bring them to the client, but essentially, we just dropped them on over. Then you run your Gatsby build command, which is going to do a lot of fancy generating and optimizing, it'll create a service worker for you.
It figures out automatically how to do code splitting effectively. It only loads the initial amount that you need, with CSS, if you import your CSS files, it'll inline critical CSS, and then load other CSS later. It does a lot of tricks for me. These are all the stuff that we're applying manually team by team before. With using these frameworks, we're able to get it automatically, which is really nice. With that, you get all these CSS and JavaScript files, and then we're ready to figure out how to host the thing. How many of you have used these tools like Netlify or Firebase or Now? For those that have used those, any testimonials you want to shout out? Do you love them? Do you hate them?
Participant 2: Yes, definitely. I use Firebase, it's a great tool.
Ferguson: Great tool. Yes. What do you use Firebase for?
Participant 2: With some simple applications where we only need a backend to host our data. Then the front end that delivered all the ...
Ferguson: Yes, Firebase is a great tool for that. All these hosts have some really cool properties, they all tie in with content delivery networks. They all like host static sites and static apps beautifully, they provide really cool features that I really wish I had for my developers, for example, being able to tie a URL with a getBranch. You update your getBranch, then it's automatically within a minute or something, your website is updated, fantastic tools. I really wish I could use them, but we can't at PayPal, we have our own internal cloud.
For a sec, we could say, "Look, we're dealing with static assets here. These are public things ultimately, that CSS and JS is going to be on people's browsers, but we have a pretty security-focused mindset at PayPal. Just being able to say, "Hey, we're building this new app, we're going to use Netlifly, don't worry about it," isn't going to fly. I'm never going to get permission for that.
Realistically, I may be able to use some of these tools down the road and I think they're great. If you're at a startup or a situation where you can, not everyone's going to do that. In our case, we had to build this JAMstack development experience, the experience that those great CDNs and host provide inside our own private cloud, which was somewhat nontrivial but has been a really fun project. The cool thing was this particular project, Superstatic and serve-handler. This is basically the open source version of ZEIT's Now and Firebase, web hosting, or static app hosting part.
These particular two tools, they helped us get mostly over the hurdle. If we wanted to run this stuff inside our own cloud, here's some of the server pieces that we could use to do that. They both offer this amazing config file format, they actually use the same format. It was started by Firebase, where you can like config set up, redirects for outdated URLs, URL rewrites. When I wanted to do something like paypal.me/star and point that to a file, I need to use this config file format to do that.
You should be able to set up course headers just in the config file just for my app. It's a bunch of really nice stuff that these tools provide. I didn't have to build that stuff myself, because I knew realistically, this is a project that I'm trying to convince people at work is a good idea, I have to build it fast. Otherwise, my bosses are going to say, “You know what, you're taking too long. It sounds cool, but it's not going to happen.” Unfortunately, what this means is, instead of being able to have my architecture look like this, what I wanted everything off the CDN close to the customer, I was presenting on the joys of that earlier, I'm going to have to run it inside our application server pool, and take the front end application that we're building, and marry that with a static app server that serves it up.
The downside of this approach is, I still have that production deploy across many servers problem. This is how we're testing right now, unfortunately, it was static apps, we can get them to work in our infrastructure, that's good, but they're still going to have to be the same slow deploy across many servers. The cool thing that we did with this to at least give developers some taste, is for staging, we're able to say, ok, you can have command line deploy, you do get on these other systems like Firebase and now, you can have command line deploy if you're using this system. We don't have that right now, it's too hard to do command line deploy with a Node app. At least in our case, you have to restart the server and do all the stuff. With this is just static assets, you can upload those to a server pretty easily, we're staging, we are able to get fast uploads.
Static Apps Adoption Plan
Our plan for bringing this into PayPal was let's just add some basic static app capabilities to our current build and deploy pipeline. This all comes down to how do they sell this to the bosses. We had two varying approaches when we started out in this process. One was, we want to just use Cloud Functions at the edge, it's a really cool idea, but it's a multi-year project. The other one was, let's inch our way into static apps by letting them work on our current cloud, and then building out a static app service that will let us do it better. With that, we're hoping that once we get the static app service internally, developers can ignore the app server, they just deploy once, your production deploy is down to something like one minute. The downside for me is now the infra team has to maintain this whole thing. Hopefully, we can keep it going.
That approach to static app service looks something like this, I've got my static app server. For every site, PayPal.me summary home, whatever they are, when a request comes in, all you have to do is consult this app mappings database. I haven't built this yet, if this is super naive, if someone knows better, please tell me now. My thought was, we just hit this database with this manifest that says, "Ok, you're the slash summary app. Your static files are living over here in cloud storage over in this bucket and this is where you find them." All the teams just can share. We point all the apps, all the load balancers to this one application server pool, which I maintain, and then all the app development teams can just go, and when it's time to generate to build their app, they just generate their files using Gatsby or whatever tools we decide. Then just push them up into the cloud and they're deployed, and then we have to update the mappings and the mappings database. Then request that comes in, as far as we can tell, should automatically just start pulling in the latest data, because we'll be serving the data through our app servers from that cloud, we can catch them, the app service level.
This is the thing that we're trying to build, I think it's going to be so fun. The coolest thing about this for me is rollback. How many of you have had to do rollbacks before when you've deployed code? Pretty much everyone, most of you. I don't know if this is normal, but if you have a lot of servers, sometimes rolling back takes a long time. It's the worst thing, "Hey, I just broke everyone's site, let me very slowly unbreak your site." Nobody wants that. With this, you can have rollbacks almost instantly. All I have to do is update the app mappings and say, "Hey, don't use this data, use that data." Then it will start pulling that stuff in right away very fast. It's very good, not for just our developers, but also for our customers.
Because we're running in our cloud, we've missed out on the CDN aspect of this thing, our like longer-term strategy is to either stick the CDN in front of the whole static app service and let it soak up all the static assets it can or potentially push this logic out to our edges. There's a lot of optimization stuff. If we can get here, I think people, both customers and developers will be very happy.
Retracing Our Steps
Conclusion time, people, hopefully, this has been insightful. First things first, when that gets too hard to do the right thing, when you keep having to reinvest in training because every single new team coming along makes the same mistakes over and over again, maybe you should think about an architecture change. This is what occurred to me doing five years of Node and it seemed like we kept teaching the same things over and over again. I was, "Maybe it's not them. Maybe it's Node." Sorry, Node people. Maybe I'm wrong about that, who knows.
For us, we found that a JAMstack provides the stable and perform the architecture by default. You don't have to think very hard about it, you just get it. You can totally screw it up, you can screw up any system if you want, but it should be like most people developing things without thinking very hard, are going to get some really good wins. Again, we are focusing on static apps instead of static sites because who really uses static sites anyway.
We think our developers are going to totally love this thing. The developer experience of using tools like Gatsby and not having to worry about the whole server level is really nice. Remember, we shrunk our debugging surface area quite a bit. As an application developer, when I push out new code, I can be a lot more confident in it because I'm literally just pushing up some files, which I've already tested the exact same versions of locally. Even though it's going to be some work to build this infrastructure, I think it's totally going to be worth it. I think if you take the chance on this thing, you're going to love it too. JAMstack is going to be huge.
Questions and Answers
Participant 3: Thank you for your talk, it was great. I've been looking into Gatsby myself as well, a little bit, and it's really nice piece of software. Something very closely related to the JAMstack talks is the cost of JavaScript.
Ferguson: That sounds mean, why are you making fun of JavaScript?
Participant 3: Basically, the idea is that, like you mentioned before, you have one and a half megabytes of JavaScript that you're shipping to the client.
Ferguson: Yes. The cost of client-side JavaScript, very heavy client-side JavaScript.
Participant 3: Exactly. Are you also looking into reducing that?
Ferguson: Yes. The cool thing about Gatsby is that it automatically code splits a lot of things for me, it splits my bundles out quite a bit. Whereas before, manually doing this stuff with the Webpack, we didn't have massive bundles. Since we started using Gatsby, we found these things are way smaller by default, because it even does stuff like it will preload the stuff when you're going over a link. It'll preload the files needed to serve that link, it does it automatically. My answer is a cop-out answer, which is, I'm really grateful that Gatsby is investing time in this because I don't have the time to invest in that. That's the best answer I can give you on that subject.
Participant 4: Have you got any tips for us where, we've done something similar over the years, but you find the frameworks that you're using, the JavaScript frameworks, like Angular and so on, it's just getting more and more weighty. Are you advocating for a multi-stage load then and load this stuff in pieces?
Ferguson: Like I said, we're really happy with the Gatsby tool because it thought about performance a lot by default. No, I don't have a good answer, man, you figure it out, you give the talk. We haven't quite solved that problem yet, sorry to say, it is an interesting thing, but again, it's hard to break up those JavaScript into smaller pieces. Luckily, we chose a framework that does it in a couple of years. I just keep hoping that these framework makers will keep making it more optimized, but I'm not that person, sorry. I feel bad about that one.
Participant 5: You said that Node.js server-side rendering was low and it took a lot of time to do the rendering. Can we compare the impact of using React and JSX versus using something that actually spits out only HTML, instead of generating a full dumb element?
Ferguson: I'll try and summarize his point because I think his point was - you know what, you're complaining about Node, but probably you're just doing Node wrong, not even being mean. His point is React is very slow to render. Before we were using React, we were using Dust to do our rendering. Dust is very fast and very buggy, in our opinion. We switched to using JSX and many applications, developers are much happier. Maybe it's going a little bit slower, but the true slow down isn't just the rendering, because the rendering I'm guessing maybe 50 milliseconds, something like that, it's not great, maybe 100 milliseconds. When we're doing the HTML rendering, we find ourselves having a ton of middleware that's getting executed, that's not even necessarily involved. Just the way that we build Node apps, sort of with all these middlewares, the way that they get stacked up. This is very specific, but whatever. You might know what I'm talking about. We find ourselves loading a ton of services and doing a lot of things that's not really needed. We find that our Node apps get bloated very easily. I think the bloat of all those service calls and middlewares are actually what makes it slow. Yes, we can optimize it a ton, but I think it's more about how it's easy to do the wrong thing, rather than yes, of course, we can make it faster.
Participant 6: Quick question about the GraphQL layer that you guys have, how much of the performance gains or the ease of the process do you accredit to adding a layer of GraphQL in between instead of calling each of these backend services asynchronously?
Ferguson: A lot of it comes down to internal PayPal things, we have a ton of microservices. Sometimes they're not documented well and having a schema, like a centralized place where we can just go check the schema, see everything that's available is really cool. That being said, the stuff I showed you is somewhat experimental within PayPal, it's not like this is widely rolled out. I will say I loved it, I think more people are going to love it, but I think the GraphQL story of PayPal is yet to be fully written.
Participant 7: I do not understand, when we use a CDN, often we will cache things forever. While with JavaScript, we can use cache busting, we can avoid the problem of wrong things caching, but with HTML, we cannot do busting because the URLs have fixed it. How you solve this problem?
Ferguson: Ok, two things. One, all the assets, they get loaded beyond the HTML, the CSS, the JavaScript, those are all hashed and they all have their own name. Essentially, it's just that initial call for HTML. What I'll say is this, static site-specific CDN, like the Netlify's of the world, and these other ones, they already accommodate for that. They have a way to account for that so that the HTML pages don't get cached with the other assets do. For us internally, we, of course, all of our HTML assets are actually being served through our - it's actually a Node server such as I was dumping on Node early, we're still using it because it's great - are actually being served through a Node server, we don't have that problem.
Moderator: You mentioned that you couldn't generate all the profiles statically, are you sure?
Ferguson: Can I generate a million profiles? Yes, I could. I haven't measured how long it will take it and I'm scared, too, to be honest.
Moderator: Measure it.
Ferguson: I'm scared, it's so many. Actually, the Gatsby team, they're working on some interesting features where you generate all the stuff, let's say you have a ton of products, a million products on your page, you generate them and it automatically will just regenerate them when changes are made to a specific one. I mean, there's ways to think about this. These problems are not fully solved, but I'm telling you, static apps are absolutely the future.
See more presentations with transcripts
