Transcript
Woodruff: I'm really humbled to be here because it's my first time at QCon. I know this is very senior, so be gentle with me. I want this to be a conversation. I always get a lot of good insights when I do this talk. So if people have ideas, please pass it along. I've had actually people do pull requests. All this source code that I do is all on GitHub. People have gone in and done pull requests and given me ideas. Hopefully, this is a two-way street. So we're going to talk about ASP.NET Core 2.1 in terms of web APIs.
How I got started with building APIs was about seven, eight years ago. A few guys and I, we were Microsoft MVPs at the time and we were building an application for the MVP summit. We built an API in the back end, and we built a Windows Phone 7. If anyone remembers that platform, it was actually really fun to build the apps for it. No one used them. But at least it was really fun to build the apps. I really got a passion for building good APIs because I did build a very good API for that app. I started digging into learning more about HTTP and learning more about REST and learning more about data caching. So we're going to really talk about stuff that I've learned over the last seven, eight years. Hopefully, it'll hit a note with you guys.
Chris Woodruff, I am a developer advocate for JetBrains. I cover all of North America for .NET. I cover primarily ReSharper and Rider, Riders .NET Cross-Platform.NET IDE. So I'll be using that tool. If you go, "Hey, what's that IDE?" It's Rider. At the end, if I have a couple people that I remember at the end, I'm going to give away a couple licenses to either ReSharper or Rider. It's up to you what you want.
What Do You and I Do Wrong?
I always start this out with what you and I do wrong. It's really what I did wrong. And I think I went over that a little bit in the beginning. I just started getting and started doing APIs without really thinking about them beforehand. And that can really get you in some trouble. Let's talk about what we do wrong and maybe our teams, because maybe you're more senior, but you have people that work for you that you have tasked to do certain jobs. They might do these things, so maybe you can go back and remind them not to do these things.
The first is the dreaded connecting your controller. This is all MVC pattern with ASP.NET Core. We've all seen the demos Microsoft has out. And they're really good demos to learn. But you shouldn't have a connection between your data access and your controllers. It's like saying you should have all your code in one project. It's probably not a good idea. The other one is the dreaded spaghetti code, or having lots of code in one place. I like to spread my code out into logical compartments, so one, it's organized well, and it's also testable. Now, at the end, I'll talk about how I test my APIs in both unit testing and integration testing, which is a new testing technology that Microsoft came out with, with ASP.NET Core 2.0.
One thing that I always got burned in the beginning was coupling my domain knowledge to my data access on the way I build my APIs now. And really, I don't connect anything up and I don't couple anything. So my domain doesn't care and doesn't really know where the data is coming from, which is really important when I have to do testing and I have to bring in a new project that contains mock and maybe mock objects that I'm going to use to test my stuff. My domain objects don't care.
Back eight years ago, I think we do more thinking about testing now. But who honestly didn't really do a lot of testing seven, eight years ago? You can be honest. Not very many of us. And we're learning. So before you start architecting your APIs, you really have to think about how you're going to test them. I always say there are two ways that I test my APIs: internally with unit testing, and externally with integration testing, and we'll look at that like I said in a little while.
What Does ASP.NET Core Help Us with?
Most of us or a good number of us were at the previous talk before lunch where they talked about the CLR and the CLR core. But here's a question I'm going to ask you guys. So maybe you guys will get over food coma from lunch. Who uses ASP.NET Core? A good number, okay, awesome. Tell me what you guys like about ASP.NET Core and .NET core in general?
Participant: Portability.
Woodruff: Portability, so cross-platform. That's awesome. So that way you can put it on any platform. You can put it in containers. You can use Docker. That's awesome, anything else?
Participant: Templates.
Woodruff: Yes. That's good, templates.
Participant: System.Web.
Woodruff: Oh, yes, the dreaded System.Web. I think Jared hit on that. When they build .NET Core, they really could allow that baggage go. That's really awesome. There's one thing that …
Participant: [inaudible 00:07:08]
Woodruff: Yes, that was a thing. When I started digging into .NET Core and ASP.NET Core, the thing that blew me away was dependency injection that was built in from the very start. I understood dependency injection in .NET framework, but man, it was difficult to use some of those third-party libraries and tools. I had a hell of a time sometimes using inject and stuff like that. Also, I had a hard time when I built those solutions. Maybe junior developers or developers that came in, they had a hard time understanding those projects and getting up to speed on them, because the code was sometimes a little convoluted, so to say. But I love how dependency injection was built in from the start. We'll look at how they leverage that especially with testing and using multiple data access projects to do your solutions.
HTTP Quiz
I do another talk at places, like I'm doing it at VS Live in Orlando. I do a talk just on HTTP. I guess I'm kind of crazy. I love HTTP. I think anyone who builds Web APIs or APIs in general should at least understand some of the concepts of HTTP. We can call it REST, but you should just know some of the stuff. So I'm going to do a little quiz. Okay? To make you guys wake up and give me some talking. I'm going to give you a resource and I'm going to give you a verb and I'm going to give you the expected outcome of this HTTP request. And I want you to tell me what the response code's going to be. Let's see what you know. I have a resource called "Products," and I'm going to use the Get verb, and the expected outcome is I get a list of all products that are in the system just through the API.
Participant: 200.
Woodruff: 200. That one's simple. That was like a softball. Now I'm going to give products, but I'm going to give a parameter for it where the product color is green. And what do you think the response code for this is? 200 again. I can't fool you guys. So it's pretty easy. Now we're going to get products and a post. See, now we're getting different answers. I think it's a 201, because 201 is created. Now, you may disagree with that because you may say, "Well, this 200 is more applicable." I'm not saying you're wrong. I'm just saying I want you to think about what you're going to send back in your API through the HTTP response. Now I'm going to look for one specific product and it has a key of 81. What should I get back or what should I expect?
Participant: 200.
Woodruff: 200. Now I'm looking up something that doesn't have a key in it.
Participant: 404.
Woodruff: That one's simple. We all see 404 a lot and day-to-day, so not found. Now I'm going to do an update. Now I'm going to do a put on a product that has a key of 81. So we know 81 exists. What do you think the response code's going to be?
Participant: 200.
Woodruff: I hear 200. I hear 204. Anyone think anything different than 200 or 204? So 204, no content. So why would it be no content? Anyone want to guess why I would do a 204 and have no content?
Particiant: You wanted to be more [inaudible 00:12:06].
Woodruff: Yes. I want my responses to be as small as possible, as tight as possible. I've already sent over that object. All I really need to know is it got updated. I'm going to go back. You got me on this one. Now I'm going to call that again with the same put right after. I saved it, and maybe 20 seconds later, I send another put over. You guys already saw the answer. So it'd be a 304 not modified, because it hasn't been modified. I'm sending over the exact same payload, the same object. So, say if it's JSON that I'm sending over or XML, and it matches, I want to communicate that to the consumer of that API, just say, "You know what? I didn't do anything with it. It has been modified. But I'm at least acknowledging that I'm going to send something back to you." The last one is deleting a resource. What do you guys think?
Participants: 204.
Woodruff: 204. Yes, no content, because if I'm deleting something, why would I send it back, right? All I really need to know is I'm going to send it back. This quiz was really supposed to make you think about how you want to communicate. I always say HTTP is the language of the internet. How we communicate with our APIs are very important.
I have this little workflow that I kind of figured out, I sat down and went through. It gives you this thing that says yes or no when you move through, and you can figure out what kind of response code that you want. I'm not going to go through all this. I really just put this up there. The slide deck will be available for you guys to download and take a look at. Actually, it'll be up on the GitHub repository after this talk also. We build our APIs, but then we have to figure out how we're also going to communicate using HTTP.
The other thing with .NET Core that's really important in ASP.NET Core, is to really learn about asynchronous development. Now, we're all senior, you probably all know this, but it is really important. I think at some point, .NET Core may not let you do synchronous programming anymore. It just may throw all that out and just go, "Everything has to be asynchronous." So I'm going to hedge my bets and just do as much asynchronous development. I do that quite a bit, or I do it as much as I can in my project. So it's just a reminder to do async, async/await. We do it a lot in the JavaScript side in our clients and the consumers of our Web API. So we should do it in our APIs also.
A Look at the Architecture
Let's take a look at the architecture for my APIs. I have four parts to this. I have an API project. And each one of these blocks within the triangle is a project in the total .NET solution. So I have an API project. I have a domain. Why I put it in the center is because it's kind of the linchpin for everything. And then I have data, and then I have tests. I always try to stress that tests are just as important of a project as any other project in my .NET solution, in my .NET Core solution.
Let's look at some code. I have this and I've set this up, and hopefully everyone can see it. You can see that I have a number of different projects. Again, we have our API. We have our domain. We've got data. And then I have a number of different tests while I also have a project that contains my mock data. That's important because I don't ever want to test, do unit testing with production data. I have a really simple mock data project that we can look at. Then I have unit tests, which are xUnit. Then I also want to throw in low MS unit test just so people can see the difference between xUnit and MS unit.
So if I open up my API. We all know this, the new startup.cs, the Startup class. You'll see that I have a lot of stuff. One suggestion I have is to keep this small and tight. Break out your configuration. Try to organize things a little better. That's a little help. The other thing is I use Swagger. I don't know if anyone else uses Swagger, but I'm going to show you how easy it is to set up Swagger in your Web API. All you really need is Swashbuckle.AspNetCore for your new NuGet packages. Iin your configure services, you just add that SwaggerGen. You can give it your title and the description and the version of the Swagger doc.
Then down here in your configure method, we're going to use Use Swagger and then use Swagger UI. So, you can also see I've got some core stuff. I'm not going to go into cores. I'm not going to go into identity in this talk because those are pretty complicated subjects and I don't have that much time. I'm going to run it just so I can prove that this actually runs and my Web API is functional. So you can see that just by doing that small amount of code, I get Swagger built in right from the start. Now, I do have the raw machine-readable swagger.json that I can have other libraries pull and create clients for me.
But just to prove that this is all working, let's just go out and grab all the albums. So the database I'm using for this is called Chinook. Chinook was a test database, a demo database that Microsoft created a number of years ago, and it simulates an online music store. Why use this one? One, it's a little smaller. And two, it's not bicycle parts. North wind and venture works are pretty boring. They're fine databases if you need something big. But for like these Web APIs, I just like something that has a little more interesting data. You can see that in here, if I increase, you can see that here's an album for those about to rock. We salute you. That's a AC/DC album, "Balls to the Wall," another AC/DC. It's a little interesting. Out on my GitHub repository, I do have a link to where you can get this database that you can use with the Web API project. So you can get everything to run this.
API Layer
Let's go back and one, let's take a look at and talk about the API project. So we looked at documenting your API. Let's talk about the API layer. It's really important. It's the thing that kind of communicates and gets your HTTP requests and send backs and then it will send back the responses. I kind of equate it to, if you're building ASP.NET MVC application, your UI should be thin and dumb. These controllers that I have in my API projects aren't very complicated. They don't have very much code to them because all I'm really doing is getting something, getting that request in, I send it back into my domain to be processed. I get something back. And based on what I get back, I'm going to send a certain kind of request or a certain kind of response code back from my API to the consumer. So, it doesn't know anything about domain. It doesn't know anything about the data access, which is really important.
The next one is I like to interact with my consumers using ViewModels. Now, I call them ViewModels. You can call them whatever you want. But they are not the entity models that are generated from entity framework core. Those are a good starting point. But I like to have a little more flexibility with my APIs so that if I had something that was different or maybe an object that I'm going to send back that was maybe a mash-up of two different entity models from entity framework, I can do that. In the beginning, when you're first learning to build Web APIs, you basically will get entity models back, send them back, and you'll send them out to the consumer. You'll assume that that's what you want back from your APIs.
But think about this example. I have albums and I have artists. When I send back albums, if I send back one or a set of albums, the album really is just an album ID, an album name, and it has an artist ID in there. So when I send that back, that doesn't really give me all the information I really want to send back to the consumers. I want to send back the artist name with it. But if I send back from the API in a HTTP response only the entity model of an album, I'm locked in to just what is in that object based on the database table. So that's why I think ahead and say, "You know what? I'm going to build out a bunch of ViewModels." They may be 90% the same as my entity models, but at least if I have any changes down the road, I don't have as much technical debt and I'm not going to have to change and cause a lot of issues in my projects. So just a thing to be aware of when you're building your APIs.
Domain Layer
Domain layer. I'm going to go through this first and then we'll take a look at the domain layer code. This contains my entity models. It's containing my view models. In my domain project, I don't have those entity models exist in my data projects because I want them in one single place. I want all my data projects - if they're production data or mock data or another, maybe I have a SQL server project that I can get data out of a SQL server database, and maybe I have another one if I have to move over to MySQL or Postgres or Oracle - but by having the entity models in my domain, one, I'm keeping those in one place. I'm not having duplicate code running through my solution. And two, if someone comes in and develops a new data project to get data from a different database, they'll know what the entity models are supposed to be.
My domain layer also contains all of my interfaces that my data repositories are going to adhere to. The thing with the dependency injection with .NET Core is it kind of forces you down the interface path, which is good because I didn't do interfaces as much 10 years ago as I do now, AND I really suffered because of it. So all my interfaces, my contracts, are in my domain project because that's where all of my data projects have to come and figure out how they have to implement. So they all work at the same level.
My domain layer uses dependency injection for all of the repository. We'll take a look at how we do dependency injection real quick. Probably most of you guys already know, but for the few people that don't, we'll take a look at that in a second. Then I think of my domain layer as my supervisor. So my supervisor is an actual class that's in my domain layer. It takes in view models on one side and will spit entity models out the other side and vice versa. It's really where I do my conversions, I do my validation, I do any error checking. It is really where all of my work gets done in my solution, because I don't want that done in the data access, because all my data access project should do is get data and push data back and forth from a data repository. My APIs don't need to know any of that, even though you can decorate your controllers with attributes to do some kind of validation. I've mixed reviews on that. Sometimes I do it, sometimes I don't. I'm moving against using those validation attributes in my controllers, and pushing everything into my domain so that those controllers stay as dumb as possible, as simple as possible.
Let's take a look at my domain. Well, first, let's take a look at how I do dependency injection, just so people know that have never seen dependency injection in ASP.NET Core. In here, I have a method called, "Configure Repositories." That gets sent in the IServiceCollection from my startup class. In here, I'm just doing a bunch of AddScopes. So this is how we inject our data. I inject my data repositories into this project. You can see that it takes an interface and then it takes the class that implements that interface contract.
I showed you that because, well, one is there's all my repository interfaces. These are pretty simple. It's just a bunch of methods that simulate all the data access method. Basically, it simulates all the HTTP verbs that I want to do. So I can do my gets and my puts and my posts and my deletes. Then I can have multiple gets if I want to get all or get one, so pretty simple stuff, nothing that difficult. But my supervisor, you can see that this is broken up into multiple files. It is one big class, that I do partial classes, I do partial on, because it got too big and I wanted to have a little more control over it. But as you can see here, we have a number of private variables that represent each one of our data repositories that we injected into our startup. You can see that the way that we get those objects back out using dependency injection is by calling those and getting those through the constructors of classes.
This Chinook supervisor will go out, and in the dependency injection container, grab those objects out. The reason why we had to do the interface is because they have to be pulled out based on an interface type. So, I just want to make that clear. And then if I go down the album, you can see that my album stuff isn't really that complicated either. I mean, this isn't a complicated solution. I talked to a lot of people that aren't as senior as all you guys. I kind of built this for the least common denominator developer out there. But at least it gets the discussion and gets the ideas across.
Now, we do have these converters. I built these converters just to show how we can convert between the view models and the entity models. Now you guys will say, "Well, why don't you use something like AutoMapper?" Who uses AutoMapper? Awesome product. Jimmy Bogart is a smart dude, and I tell him that every time I see him. There is a ramp up time to understanding that product, and I didn't want to throw that complicated product into this demo. Now, I may do a version of this with AutoMapper or I'm going to be doing another talk next year around fully domain driven development web APIs. So I'll probably use AutoMapper with that.
But just know that these converters are just used to convert back and forth, because I didn't want that code, that logic, to be in the supervisor so that I could at any time replace these converters with something else like AutoMapper. I'm always thinking about when I'm building my APIs and building my solutions, how to break out my code in small enough chunks because I'm not a really smart guy. I mean, you guys are probably a lot smarter than me. I'm just a poor kid from Flint, Michigan, that grew up in a trailer park. I had a professor in college that always said that. He had a Ph.D. in astrobiology, and he was a smart dude, but he was this poor kid from northern Michigan from the Upper Peninsula. And I kind of related to him. I grew up the first kid going to college in my family blue collar, auto workers. I try to keep things simple, because that's kind of me. So that's why I kind of break out this code into small enough chunks so I can come back six months later and understand my code again.
But in here, you can see that I have my entity. These are my entity models. These are my view models. So here's album; this was generated using Entity Framework Core scaffolding-Db. You can see it's a typical POCO object that represents an album from my database. It has album ID, title, artist ID. But if I take a look at the view model, I have an additional property for that artist name. Now, you could have something where there's a person table and an address table, and a person can have multiple addresses. But there's a primary address that's set in that address table. But when you're pushing a person out through your API, that person, you may go, “I want to have the primary address incorporated into that person class that goes out through my API." So that reinforces why I'm using view models in addition to my entity models.
Data Layer
We will jump back in and do the last. So my data layer, pretty simple. It's where all the heavy lifting for data happens. I have everything to find on interfaces. They're based on my domain layer. Again, if I build and I do have two different data access projects in the solution, one production database, one mock data, but they all adhere to the same interfaces so that I can use dependency injection. My API and my domain don't really know which is which.
When I run my tests, I dump in my mock data project into the dependency injection, and it gets used from the domain without knowing anything's different. Then I use Entity Framework Core 2.1 to do all this. But you can use anything that is .NET standard 2.0. And I'll probably lie. You guys heard Jared talk about what .NET standard is. It's really a standard that encapsulates all the APIs that you can use in your projects. Last, I've talked about it, but I do have a mock-up from my data layer. I just threw my favorite form of data up on the screen, so it kind of ages me.
Caching
The other thing is, up to about three months ago, I didn't have this section in this talk. So caching. It's really important to think about caching way ahead. Before you even start building your APIs, you have to ask yourself, one, do you want to use caching? Two, what type of caching are you going to use? Because how you're going to push out and how you're going to use your API will give you a different caching story. There are three types of caching that we can use with Web API. We can use response caching, in-memory caching, and distributed caching.
Now, if I was building a Web API project, and this was going to exist on maybe one server inside my company, and it was going to be used for maybe a department application, and maybe 20 people were going to use this API or the applications that consume the API, I would probably use in-memory caching because it's inside. You build it inside the code of your projects or of your solution. But if I was building this API to exist out in the cloud or in a data farm, in-memory caching would be awful to use because I want to have a caching story that caches across all the instances of my servers, of my Web API servers. That's where I would use distributed caching.
But first, let's take a look at response caching. Anyone use response caching in their Web APIs? The response caching is an HTTP 1.1 specification. Basically, it just says that if I set this attribute on my controller or action in my controller, that the consumer of that API can cache the information coming back for a certain duration of time. In this case, it would be 60 seconds. The response header that you would see if you took a look at that through Fiddler or Postman or something is you would see that cache control. And then there's actually a response cache. There's an attribute that says to enforce no caching if you did want to have explicit yes or no caching in there.
Response caching, if you’ve ever hit a API that has response caching, maybe you grab something in Chrome, and then a couple seconds later, if you call that API using a get verb, again, and it's really fast, well, it's because Chrome is built to understand and use response caching. So it actually stored that information inside of its internal memory, and then instead of going out to that API, it said, "Oh, this is cached. I'll just grab it and give it back to you again." If you do this, do this with the understanding that the consumers of your API don't have to use the caching. You're just saying, "If you want to cache this, cache it. If you don't, you don't," but it's up to the consumer to handle it.
In-memory caching. I probably don't have a lot of time to go into this. If you scan through my solution, you'll see where it is. Basically, you set it up in your startup, and then you can set it up to where you can push things into a cache in your data access and then get it back out so you don't have to call the database. If you want to dig in more, there's lots of documentation out on the MSDN site.
Now, distributed caching is more complicated. It's an external cache that you set up in your solution. This just goes in a lot of technical details. But basically, what we're doing is we're going to be setting up maybe a Redis cache or we can set up a SQL server cache if we want that. And then if we're out on the popular cloud platforms, each one has their own distributed caching product and service. So Azure has Redis Cache. Amazon has ElastiCache, and Google Cloud has Memcache. These are just different products that you can use. It's kind of interesting.
A lot of people will say, "Well, why do you have SQL server cache if you're getting your data from a SQL server database?" But your API may be getting data from a SQL server that is not in the same location, physical location. So maybe it's 1,000, 2,000 different continent and you have a cache in a local SQL server that you can collect that data and keep it much closer to your API implementation and save on some latency and some performance.
Testing
Last thing, testing. I have two types of tests that I have in my solution. I have unit testing and integration testing. Unit testing, I'm not going to go into that. We should all pretty much know what unit testing is. Integration testing is for API endpoints. Anyone take a look at implementation, or I'm sorry, integration testing for Web API? It's very cool. I want to take a look at it. You set it up very much like a unit test. If I open up my integration test, in here, I have this code and I have to have some NuGet packages set up in here. But you can take a look at that through my project. But you have to have these two namespaces, so Microsoft.AspCore.TestHost and .Hosting.
What this is, is I'm going to set up. In my constructor, I'm going to say I'm going to have a new test server that basically sets up a new web host builder that will use my startup in my API project. It uses whatever environment I want. In this case, I'm going to use the development environment. It gets this server. This is an actual little tiny web server that gets spun up that encapsulates your product, your API, the whole solution. And then I'm going to create a client based on that server. I'm going to store that in a private variable that I'm going to use in my test, so that client understands everything about the API and understands all the APIs. It understands the shape of what view models have to be sent and what comes out. It understands everything.
What I can do now is I can set up a test. For this, I'm going to say AlbumGetallTestAsync. I'm going to create a new HTTP request that's based on the verb that I get sent into this attribute. And then I'm going to call album. So basically, this is a call to get all albums. Then I'm going to send that request into the client using a SendAsync. I have two ways that I can check to see if this test was done correctly. I can say response.ensuresuccessstatuscode, or I can explicitly, if I want to check to see if it's a status code other than okay, other than a 200, I can do an assert.equal and then check the HTTP status code to whatever status code I want to the response.statuscode. So those are equal. You don't have to do both of those. You can just do one or the other.
If you wanted to check, this is an AlbumGetTestAsync. This will send in a verb and an ID. It just tests to see if that one album is in the database or coming out of your API. So remember, these tests are done externally to your project, to your solution, where the unit tests are really done against internal objects within your solution.
Let's wrap this up. I really appreciate you guys coming to this talk. If you guys have any questions, I'll be around for the rest of the day. I'll just be walking around. If you want to get the code for this, it's out under my GitHub account, which is cwoodruff and you'll find it under ChinookASPNETCoreAPINTier. And then I have an article that I wrote for InfoQ along this same subject. So there's the bit.ly link for it. You can contact me through my email @JetBrains or you can follow me on Twitter. But I really appreciate you guys coming out and giving me some of your time. Hopefully, this was valuable.
Again, if you guys have any ideas, please come up and talk to me. The gentleman here that gave me the dependency injection, come on up afterwards and I'll give you a license, and sure, Gabriel, because he was giving good feedback. And if you guys have any questions about just JetBrains in general or want to tell me anything about our products, good or bad, just come on up and talk to me. So that's what I'm here for also. Thank you.
See more presentations with transcripts
