Does Github Copilot understand streaming apps?

Using Copilot is something I recommend to every developer out there. Does it fully replace the need to understand and write code yourself? Definitely not! Is it a great addition to the developer toolkit you already use? Very much so!

The interesting thing with many developer-oriented tools out there, however, is that they often have little to no knowledge of the video streaming niche. So I took it to the test and used Copilot while writing some streaming and OTT-related code. The results…?

Video playback

Starting with some of the basics, I’m looking at creating a mapper interface in Typescript. I want to be able to play video content with both DashJS and ShakaPlayer and be able to change between the video players on the fly. So we start with our generic playback interface.

DRMType enumeration

One of the important, and very niche aspects of video playback is DRM. Let’s make a DRMType enumeration, shall we?

Demonstrated above are the steps Copilot takes me through when I’m defining the DRMType enumeration. Isn’t that neat? It knows the DRM types by heart, meaning it’s definitely capable of helping you out in that regard.

Generic Player interface

But we aren’t there yet. What about a playback interface that’s generic enough to be able to be implemented for both DashJS and ShakaPlayer? Now, for this example, I’m already using some of my own knowledge. Because I know that ShakaPlayer uses async functions for some of it’s logic, for example for loading and unloading of content. So I’ll start my Player interface similarly, by defining the first function as async. How Copilot fills it in after?

Would you look at that? It started off really strong. After I gave it the initial ‘play’ function returning a Promise as needed, it started filling in a lot of methods I do expect in a player interface. All returning a Promise, and in most cases also requiring function parameters that make sense. You’ll notice in the image above that the list does continue. And it does for a whole lot more than you can see… Copilot really seems to want to keep adding methods, also a lot that definitely don’t make too much sense. It clearly shows that you can use Copilot to quickly write code. However, you’ll have to instruct it properly with what to write and realise when you should stop it yourself.

Implementing DashJS

So now that we have our interface for a Player and a DRMType enum, let’s start with the actual implementation. For this example, we’ll take DashJS, for which I’ve already done some work myself to get set up. I’ve installed dashjs via npm (npm i dashjs) and imported it at the top of my file (import * as dashjs from ‘dashjs’), as well as importing our created Player interface. Now let’s start by initialising the player.

Ignoring all the warnings, it actually knows how to create an instance of DashJS! That’s really already a lot further than I had expected, great start. Now, the reason it doesn’t like what it’s writing yet has multiple reasons. We of course need to implement the other methods as defined in our interface. Next to that, we need to define the player as a property of the class. So let’s see if Copilot also understands how to properly type the player object.

Well, that was easy. It properly typed the player property as well. Now, there’s a whole lot more we need to do. For example, that element you see as a parameter in the init function is currently unused. It should be used in a different spot, namely the player.initialize function of DashJS (which actually sets the source URL). So let’s store that in a property in the class and then see if Copilot picks it up when setting the source.

It certainly does! You can clearly tell that Copilot has some understanding of the library of DashJS. With DashJS being fully open-source on Github, I wouldn’t be surprised that Copilot has been trained with this data. All the more reason it’s giving great examples, and being a great help with video playback integrations. This will definitely get more difficult when Copilot hasn’t had the opportunity to learn the code and interface, for example from a fully closed-source library. So your mileage may vary.

Front-end applications

A whole different beast, front-end applications. The truth is that there are simply so many ways to write your front-end applications. Of course, a lot depends on the platform you develop for (iOS, Android, SmartTV, Web, …). Even within your platform there are millions of frameworks, libraries and others to use. Thinking specifically about web or SmartTV, React, Vue, Angular, Svelte, SolidJS, Qwik,… every day a new framework, every day more to learn for Copilot. So your experience of getting help with frontend applications heavily depends on the technology choices you’ve made when setting up your project.

For the sake of experimenting, I went with a framework specifically defined for TV applications called Lightning.js. While the code is open source, it is not so heavily used in the world as React, Vue or Svelte, for example. So that’ll likely mean Copilot has had less training on code built with Lightning, meaning the suggestions might not be as good. An interesting use case that you definitely might come across developing in this niche streaming market.

A simple carousel

Starting ‘simple’, I want a carousel. A row, a swimlane, a list. The most integral part of any TV application. So let’s just ask Copilot for that, and see what it comes up with.

Okay, not great, but also not too bad. It understood the templating setup to a certain degree. It knew about the x, y, w and h properties. Shortly after things went downhill though. A CarouselComponent definitely doesn’t exist, so we basically have nothing here. Let’s try and give it a nudge by specifically telling it to use the ‘Row’ component as defined by the Lightning-UI-Components library.

After a bit of back and forth, I got a row with three cards with a simple background and title. Copilot didn’t yet understand the card existed until I typed out one of them myself. After the first one had been typed out though, it immediately caught on and understood that I wanted to write cards. It still has some trouble understanding the properties on the card, clearly showing it’s not as trained on Lightning (or the UI Components library).

Conclusions

Having only scratched the surface of app development, there is of course a lot more to learn to see how Github Copilot behaves. In larger code bases and after longer periods of time, it’ll likely get a better understanding of how you and your team write code — as well as what libraries you use and possibly also how they work.

Copilot in general though is a great addition to your developer toolchain. Being in a streaming application niche is always a bit interesting, as a lot of the open-source code is not targeted towards writing applications like that. There are a lot of specifics you have to deal with, that might not yet be familiar to Copilot. It was a great surprise to see it understand DashJS however, and even after a some nudges Copilot also understood a bit of Lightning. A great addition as a developer if I’d say so, Copilot will definitely make you more effective as a developer.

Something to always still take into account is that you’ll still have to do a lot of thinking yourself. Copilot is great at autocompleting your code and copying from earlier examples of code you’ve written yourself. But for a lot of the logic, and knowing where to write what type of code, you’ll have to rely on your own insights. Copilot is exactly what it’s name says. A co to you being the pilot.