VR Environments for Collaborative Software Development

Here you can watch the video of this podcast:

Transcript

Shane Hastie: Good day folks. This is Shane Hastie from the Engineering Culture Podcast. Today, I'm sitting down in a virtual space with the folks from Moderne who are using virtual reality in developer collaboration. So, I'm just going to let the conversation go as it goes. Jonathan, do you want to kick us off, introduce yourself and then let's see who's around this virtual table.

Introductions [00:35]

Sure. Yes. Thanks for having us Shane. It's a pleasure. So Yes, I'm Jonathan, one of the co-founders of Moderne and we're a software shop. We build automatic software repair, refactoring for enterprise customers. And we're about, a little over 10 people now, and scattered all about. And maybe you could each introduce yourself, and where you're at in the world, because you'll see that while we appear to be close together, we're not very much in reality. So, Pat?

Patrick: Yes, I'm Pat, I've been with Moderne for a little over a year now. Work mostly on the open source side. I'm in Chico, California, which is in the Northern Sacramento Valley of California. Yes, it's pretty fun.

Tyler: And I'm Tyler and I'm in Portland, Oregon. I've been with Moderne for about a year and a half. Almost two years. It'll be two years in this fall. Anyway, Yes, super fun.

Johnathan: Where are you based?

Tyler: Portland. I said that one.

Johnathan: Oh, sorry. My bad.

Kevin: I'm Kevin. I've been with Moderne for just over a year and I'm normally based in Texas, but right now I'm in Buenos Aries, Argentina. And I've been here for, what, two weeks. And I'll be here for another month and a half.

Jon: I'm Jon. About a year. I'm the other Jon. Jon as well, Jonathan. So, sometimes need to be more specific than just Jon whenever we're talking. But I live in rural Southwest Missouri, actually.

Kyle: And I'm Kyle. I've been here almost a year, and software engineer as well. I am out of Sacramento, California.

Shane Hastie: So, nice to meet you all, gents. And I'm Shane, I'm sitting in a little town called Otaki in New Zealand. So a fair way away from you gents.

Johnathan: Yes.

The immersive space [02:44]

Shane Hastie: And, it's tomorrow. And I can tell you the weather forecast is clear and sunny, and looking at the sunlight streaming into your room, your virtual room there, it looks like your room is sunny too. So, what I see is a virtual table and I'm guessing that we all see the same thing, something I also see is avatars. How realistic to you are those avatars?

Johnathan: They're pretty close.

Tyler: Yes.

Johnathan: Really if I look at any of them, I would know exactly who they are, even if I'd never seen the avatar before.

Jon: Yes. We did spend some time, probably each of us going through the various options of tweaking on the face shape and the eye shape and the hair color and our hair in my case. So I actually had to have my wife and my daughter help me because I casted up on the TV. And so there's a hundred different options for very subtle changes. And of course, I don't know what I look like, because I don't spend any time looking at myself. So except in Zoom. So I would just, this is it. And they were like, ah, it's a little more like this one. And so it was a somewhat time consuming process. But in the end I think it got reasonably approx, approximately close.

Shane Hastie: Why VR, everyone else is just going with zoom and we are also seeing asynchronous coming into play. What made you go down the path of VR?

VR feels more personal and engaging than video calls  [04:26]

Johnathan: I think honestly we use all three of those. We do async, we do sync and then we have different tools for different things. And honestly I was still skeptical when we first said, Yes, we're going to go into VR. I'm like, oh okay. Kind of history, but let's try it. And honestly it's hard to describe until you actually sit in front, you sit in one of these in Oculus on your head. It is different than zoom. It feels, Yes. I don't know. More immersive, more personal, more personal it. Yes, we don't do it all the time. We definitely use zoom a lot too, but just the fact that we're able to use the whiteboard and stuff and we'll have to show that off. That's kind of interesting because we don't have the opportunity to do that since we're all scattered all over the world, basically. So from our experience, there is spatial audio.

So Tyler does sound like he's over to my right now. And so it's a bit more immersive in really subtle ways that you know, are used to a more flat zoom experience. I noticed one of the first team meetings we did in here, Kyle, I think was sitting next to me. And normally if we're on zoom, nobody knows if you're doing something else on your laptop at any given moment. But here when Kyle was speaking, I actually had to face him or he could tell that I actually paid attention to him. So it's just kind of interesting, the little things that get lost from, in person over those sorts of environments.

Shane Hastie: So now I'm not present in your virtual space. So how do I appear to you?

Kevin: You're on a little screen. It's kind of like you're in a meeting room, we're in a meeting room and you're remote. And so you're meeting with us remotely. Yes. You're projected on the wall out there.

Tyler: You're on a floating TV.

Johnathan: Fortunately. I think that means you can't see the beautiful ocean behind you.

Tyler: Not yet.

Johnathan: We can change that.

Tyler: Yes. I think we can change the, actually change the configuration.

Johnathan: Move them around.

Changing the view and presentation mode[06:36]

Tyler: Yes. I bet if we change to the presentation mode and so there's different modes actually. Well I think you could probably see the ocean behind us now. Yes. Yes. So in this mode it sets us all up on, in a row where we can all see the whiteboard in front of us. If somebody was presenting, we can share our screen to the front whiteboard as well. We can show that in a little bit. So there's this mode. If more people join just another row will appear behind us. So the room scales, according to the number of people that are in here to try to give it that cozy feel at any given time. So I know unless you one time. Yes, go ahead. Unless you're the one person that ends up going to the second row, then you're the only person down there.

Kevin: It doesn't rebalance. It just puts one person back there, poor Aaron.

Tyler: And then this is the last one. This is almost the more formal board table sort of view of the world. And one thing I will say that's different about VR is right now, even I'm kind of a personal space guy and Jonathan's a little bit too close in real life. I would never allow this. Right. It's too close to me. And you get that impression. It's the feeling a little bit of someone in your personal space. I think there's actually a setting now or there will be a setting in some of these that's a personal space protection thing where if one of the avatars starts to get into your personal space, that it'll automatically keep them out of that. I wish I had that in real life sometimes with me every day.

Shane Hastie: Well let's geek out a little bit on the technology. Tell what is the technology that is running all of this and doing this for what's involved in setting it

The technology in use  [08:35]

Tyler: I was going to say the barrier to entry is pretty low. You just have to buy in Oculus and then honestly there's some time setting it up. There's going to a time commitment. And I think all of us probably spent an hour or two setting things up. So that was kind of a hassle. But once you do that, it's pretty straightforward and that's really it. I mean, Yes, Oculus and there's plenty of options. The way that we chose to do VR is using the quest two headset in particular. And this is their workrooms application, which is on the quest. And I don't think it's available on rift. So I think that you could use some of the other ones and I don't know if you can use an HTC or some of these other headsets. So it's pretty specific to Meta’s headset.

But if you're there's other applications out there that do this sort of conference room or collaborative type thing, and you could use those applications with other headsets than the quest. So the thing that I like about the quest is that it's all self contained. And so even though it's heavier and because it's got all of the processing is happening in the headset, which is limiting on the amount of processing power it can handle and the number of objects and things that it could track at any given time, which is why you would use a more powerful PC based on for doing an intense gaming or something like that. But for doing collaborative work and being able to see avatars and get that personal feeling, that's different in VR than it is in a 2d zoom, then you don't really need a lot of stuff.

But then if you wanted to do more than that, if you wanted to do in depth screen there, there's beyond just collaborating like we're doing now and talking to one another, there's the ability to you could actually work in VR. If you wanted to put up a screen and you wanted to have multiple screens and you wanted to have browsers open and things like that, you can do that. And there's different apps that do it in different ways, but it requires more set up on your part. It requires in some cases, more computing power.

Johnathan: So it's sort of like, what do you want to get out of it? Exactly. So I think one thing we probably haven't made clear is that we actually have our laptops. So these keyboards that you see on the screen in front of us are actually our keyboards, but positioned, virtually exactly where they are physically with respect to us. So the cameras on the bottom of the Oculus are actually looking at where our keyboards are physically and positioning the virtual one right there.

When we look down at our keyboard, we see a pass through to our hands sitting on top of a virtual keyboard. So your hands move at the same rate they would because it's just a camera passed through to your hands. And then we see a virtual screen in front of us and we can make that screen small or large. And any one of us can project our screen onto the front. I don't know if somebody can do that. My computer is connected right now, but Yes. So this is a projection of Kevin's actual desktop on his desk up to the screen on the front so we can do sorts of things so we can do of laser pointing and this kind of thing, pointing different things out here and that's part of it.

Kevin: And then I'm, I don't know if they mention Yes, well you have to use the whiteboard because that's really cool. I don't know if they've mentioned yet, but the headset itself only $300. So a lot of the VR headsets that you can get are much, much more like up to a thousand dollars, but these are 300 bucks a piece, I think really a low barrier to entry.

Johnathan: Let's maybe show the whiteboard because that's kind of an interesting feature and something we miss, I think when we're doing remote work, typically Kevin's our going to be our Guinea pig. Yes, here we go.

Kevin: Well, unfortunately me being the first Guinea pig means that my big thing here is surprising people, not Jon, but whoa, surprising people. So I can now immediately be in his personal space. Yes, Yes. Too close too, too close.

Johnathan: To us, we actually when we switch, we actually had to stand up in our room and walk to a pre designated place in our room that we pre-cleared to be the whiteboard. The whiteboard that's in front of us is actually just open air in our real rooms. But when we write on the whiteboard here, it provides feedback to your hand. That makes you feel like you're riding on a physical surface. So my dog thinks I'm absolutely ridiculous cause I'm just waving in the air.

But to us it kind of feels like if you just suspend disbelief for a moment, it actually feels like you're riding on a physical surface. And then to everyone else, that's sitting at the desk, they can actually bring up the whiteboard on their desk. And so I think actually drawing on his desk, but showing up over here on the whiteboard.

Kevin: And if you could see the controller turns into a marker as you get close.

Johnathan: So Yes, that's kind of neat. And this is where I'm saying we really need dog tracking because in this space here, the dog will want to sneak up on you. And it doesn't really appreciate the fact that you don't want to do something, but we can throw up.

Shane Hastie: You both have actually in the space that you're in at the moment physically you've stood up and you are standing next. Yes.

Johnathan: Yep. Yes. Yep. So my desk, oh there's a wall. My desk is over there I think. And then there's another room over here and then the whiteboard's right here. So we can throw images up here and play around with them and so forth as well. So that's something you can do as well.

Shane Hastie: I can certainly see the appeal of this type. What are the downsides?

The technology still needs to improve [15:25]

Johnathan: Downsides are, let's see. Technology still needs to probably catch up a little bit. It's a little rough around the edges at times. For instance, when I connect my Oculus to my Mac, it freezes every time it wakes up from sleep. So that's kind a problem. So I think that the battery life maybe is a good thing also cause the battery life is not huge. So it kind of caps the amount of time you're going to spend in virtual reality, maybe about an hour or so at a time. And that's not necessarily a bad thing, but it kind of puts a limit on that. The freezing thing I think for Tyler, we discovered eventually was around. If you try to connect your laptop to your monitor and then also join into the room, then there's funny things that happen with screen resolution so forth. And you kind of have to remember that the monitor that you have physically doesn't help you in here.

You're seeing a virtual monitor that you can scale up to something that's larger. And so there's that interesting potential to have, I think in other apps they have multiple monitors and things like that. So you can just start placing them everywhere and Yes, laptop doesn't really care.

Jon: I would rather the VR room be the display server. I would rather have windows where I can position them on planes in the environment than streaming my desktop as taking the pixels of my desktop and translating it into a flat image. I would rather have that where I could put them farther away in three dimensional space and make them bigger and have something up here. And I could see myself really using that, but that you could put our metrics stuff up on the wall. Yes. Yes. I mean, there's all kinds of opportunities that once you break out of this desktop, because it doesn't mean anything anymore in the VR space, your applications that run on your computer don't have the same meaning as a monitor, which your world is encompassed by this monitor.

And it's defined that the borders of it are defined by your monitor. That's not true anymore. So it doesn't make sense to carry that paradigm forward into VR, but the technology's not there yet in order to support it.

Kyle: Yes. I would say I'd just a little bit higher resolution. Cause I mean, you can make your monitor pretty large, but you're still limited to the pixels that are right up against your eye in headset. So if you're used to having, having multiple pains in my terminals and in my editor and stuff. So when I come in here, I really do things more in single pains, because it's just easier for me to see it that way. And the other thing I think would be nice though, we don't run in the, to this too often would be if they had breakout sessions, if you could break out into smaller teams like that kind of thing.

Kevin: Oh Yes. You can't even. So you see there's corners of the room. We can't use those really. We, I can't say go over there, which is a missing feature. Oh yes.

Jon: In my opinion, there are some applications where you can do that. You just can't do it here in work rooms. Okay. Yes. So there glue the work, which is on the apps thing. It allows you to move around, but it's not arbitrary. There's like spots where you can go. So it's sort of a thing I think coming, but Yes, work rooms doesn't allow you to do that. Yes.

Johnathan: And then I know, did you, the engineer, the door want to know what happened? Yes. That was what I was just actually looking for the door.

Jon: It's over here. It's there steps up.

Johnathan: Yes. Yes. We had heard that being in an early form of this. They didn't have a door like an exit. I mean we could go to a different.

Jon: Let's go to Seattle. Future Seattle.

Some interesting psychological aspects[19:30]

Johnathan: This one, Yes. Think this is future Seattle. Yes. Yes. And this one, there's actually a door over there that in the corner. And I think they said in early versions, there was no door and people would actually freak out a little bit because there was something wrong about being in a room that you just couldn't escape from. Or would you feel like you could escape from, even though we can't actually go over that door. So that's a right.

Kyle: Kind of interesting psychological phenomenon.

Kevin: Yes. Yes. We've also had the questions. What happens when you put 50 people into this room? Does it grow bigger? Yes.

Tyler: Is there a limit? I sure would like to find out.

Jon: I think that, well the limit VR limit, I mean, Yes. The limit of VR is 16 and then 50 people can join via where you are Shane. See? Yes. I wonder if that has more to do with the avatars and the tracking of all meshes and things about that.

Tyler: Yes. Limitation limited, limited computing power for parallel processing. Yes.

Shane Hastie: The thing that is slightly disturbing for me is that none of you have anything below the waste.

Jon: Yes. Yes. It's definitely, I think they're saving computing power on all of why spend all of those cycles on legs. I don't know. Not the most when you, part of it.

Johnathan: You do tend to not notice it after a while you notice it first, but then you're looking at someone and they're gesturing and there's the face. And even though it doesn't track your mouth, it only pretends it's tracking your mouth. Cause it's based on audio, not based on your actual movement of your mouth. So if I smile, you don't know because it doesn't tell unless I'm a tone of voice or something like that.

Kyle: So which would also make modeling legs kind of interesting in terms of the motion, in what type of animation they would use based on your motion. I don't know.

Tyler: And there are things when you set down a controller, your arm gets all, can potentially get twisted.

Jon: Like all backwards.

Kevin: And especially if you're holding it with the wrong hand.

Tyler: You're holding, put it, Yes.

Kevin: Put it your hands like that.

Tyler: So there are definitely some distractions if you will. So like I said, this is a, just one tool in our toolbox. We still use zoom a lot. We use slack, we see whatever.

Shane Hastie: The one tool in your toolbox, when would you choose to use this approach? This tool?

Real use in software development – whiteboard sessions and pair programming [22:23]

Johnathan: I think that the two things that we've used this for the most have been to get in a whiteboard session where we are in different locations. That's pretty nice. And then the other one is pair programming, which sounds kind of strange pair programming and virtual reality. And again, this is, I'm the biggest skeptic of this. When we first started, I was like, mm, nope, but it's surprisingly immersive. And like I said, having one person having their screen up and then you can use your pointer. And I don't know, it is actually really productive for that. So it's actually a slightly different form of pair programming that rather than just being two people, looking at one monitor with one keyboard, you just, one person puts their screen on the front and the other person actually still has their computer screen in front of them if they wish.

And it kind of goes like that. And I think it was maybe it was Pat or somebody at some point I was trying to teach you something in the IDE. And it was actually interesting. He put a screen up there. I actually walked up to the whiteboard and would just point things out, click here, click here, click here. But when do you get to be the same size as your pairs monitor standing next to it and pointing out the thing that you want them to click on? It's a little bit different, but I would say that's an interesting feature of it.

Tyler: We learned about the whole thing. I'd say we got started in this. I was a big skeptic of this too, but I've long listened to this Stratechery podcast from Ben Thompson. You may have heard this or not, buta  very rational thinker. And he had an interview once where they were talking about VR versus AR actually. And their hypothesis after seeing this actually was that VR would actually have a more immediate application in business than in the sort of personal sphere, much like the PC did originally. The PC really started in the business and then individual personal applications were developed second. And with the price point at $300 a headset, it's a third to sixth of the price of a laptop. It's pretty cheap, really for a business to ship him out.

Bringing customers into the space  24:42

It's a low cost investment to get another experience to your team. We actually had, since we do enterprise software, we work with other companies as well. In one case, we had a customer engineer join us in our room and kind of working on a problem together. And I remember him saying, can I just come back in the evening and hang out here? And so as a company that's working, especially a small company, that's doing enterprise software. The idea of actually having customer engineers come to your space and potentially collaborate together and is obviously an advantage, I would say different.

Shane Hastie: What else surprised you?

Kevin: So two things for me, the hand tracking is almost really good is almost really, really good. It's really good, except for when you cross them or whatever the other one is. Jon mentioned the mouth movements. So this figures out what I'm saying or how my mouth should move based on the sounds that I'm making and not what my mouth is actually doing. Those two things surprise me. Probably the most that I can point at Pat, I can give thumbs up. I can point at people and be like, Hey, do you have anything to say about that or whatever? And it's very good because there's like, I don't know what 12 cameras on this thing that just are used for tracking. Yes.

Tyler: I think the spatial audio was a big amazement too. Yes. And honestly, I really like the fact that if you're on your computer and trying to type and why you're, there's no way that you can be distracted and people don't know, people will see you on keyboard.

I kind of like it. Yes.

Patrick: On that same note, I actually like that for presenting in here because when you present on zoom, usually you're sharing your screen. All you see is your screen. If you have the little gallery thing, maybe you see other people, but presenting in here, you can really see people's interactions like would, if you're presenting an actual conference room, just those small subtleties, it's nice to queue off them.

Tyler: It could go up to the lectern and present from up here. I could see this being sort of hybrid for meetups as well. The presenters, whoever can join joins virtually and then everyone else joins a lot like you actually.

Shane Hastie: Obviously you are a tech company and getting a lot of value from that. We haven't given you much opportunity to tell us about Moderne. So this is the two minute who's Moderne?

Introducing Moderne [27:38]

Johnathan: Sure. Yes, absolutely. So do automated source code repair. So we started focusing mostly in the Java ecosystem and infrastructure is code, but we do a big heavy list like spring boot, one to two Java, eight to 11, J four to five, that kind of thing. So think modernizing your applications. But we go and repair the source code itself in a style preserving fashion, a hundred percent accurate.

And then we do that at scale. So our open source product is called open rewrite, the core refactoring technology and all the recipes that we write to perform transformations, our Apache license. And then as a commercial company, we apply those same recipes at scale, to up to hundreds of millions of lines of source code to affect those changes very quickly across an organization. Our belief is that software used to be built from the ground up and now it's more assembled from components than anything. And that those components are moving out from under us all the time. The APIs are changing. Things are being deprecated and removed. And if you don't keep up your software, doesn't keep functioning. So we're trying to help with that.

Shane Hastie: Thank you very much. If somebody wanted to continue the conversation, ask you more questions about this, where would they find you Jonathan?

Johnathan: I think the easiest thing to do would be to just go to Moderne.io and my calendar link is a button right there on the front page. So my calendar is wide open to you if you're interested in this or anything else.

Tyler: And then the entire team is on a slack organization. That's off of our open rewrite. We go to GitHub and go to open rewrite. There's a slack channel to join and all of us are always on that. So yep. Love to see you there.

Shane Hastie: Thank you so much, folks. I really appreciated this insight into the use of VR in reality. Yes.

Johnathan: Yes. Thanks for having a Shane.

Patrick: Yes. Thanks. Thank you so much.

Mentioned

• Moderne 
• Quest 2 headset
• Stratechery podcast