Hide and Seek: Sphero vs. Drone (Part 1)

SignalR and Backend Coding

Hey Everyone. So the backend part of games tends to be some of the most difficult code to write in the project. I’m also not of fan of backend Server code (but that’s just me). In this project I pushed out of my comfort zone to step more into the hardware back end world.

Really this tutorial is about Signal R in general. That way you can follow along and make a hide and seek game, or create your own game based on these principles and fundamentals.

Getting Started

So for those of you that haven’t used Visual Studio, that is the IDE I’m programming with in this tutorial. You can get for FREE here: http://www.visualstudio.com/ 

  1. Make a new empty project

Signal R – Backend

Create a new Project in web

 

Then, in the New ASP.NET Project window select MVC and Change Authentication to No Authentication.  Click Create Project.  Now you need to get the nugget package from the Package Manager.

Tools> Nuget Package Manager> Package Manager Console

Then type:

PM> install-package Microsoft.AspNet.SignalR

If you expand your Scripts folder you will see that the libraries for SignalR have been added.

 

Now Right-Click the Hubs folder, and click Add | New Item, select the Visual C# | Web | SignalR node in the Installed pane, select SignalR Hub Class (v2) from the options. And Create a new hub called GameHub.cs.

This Hub will be the host of your server and game logic. It will also be what all clients send their messages to.

 

Under our project create a new class called Startup.cs, by right clicking the project Add > Class.

 

 

Add the following code:

using Microsoft.Owin;

using Owin;

 

[assembly: OwinStartupAttribute(typeof(SRHS2backend.Startup))]

namespace SRHS2backend

{

public partial class Startup

{

public void Configuration(IAppBuilder app)

{

app.MapSignalR();

}

}

}

 

Now go to HomeController.cs in your Controllers folder.

Add the following code snippet:

public ActionResult Game()

{

return View();

}

 

Now this next part is mostly for seeing which messages are getting to the server and displaying them for the developers to see. We didn’t implement it in this code but there are sources online that show you the javascript and jQuery needed to print to the new view we just created her http://www.asp.net/signalr/overview/getting-started/tutorial-getting-started-with-signalr-and-mvc

 

Now, your gamehub.cs class is derived from SignalR’s Hub class. This allows you to access all the currently connected clients and the methods within those clients.

 

To understand it better, the CreateGame(User user, Game g) method would be called in the client code, and defined in GameHub. While the Clients.All.gameGreate(g, sm) is defined in the client side of the code.

The User, Game, and ServerMessage are classes that I created for this specific game. They hold information that is required by each User, Game, and ServerMessage. While AllGamesList and AvailableGames, are Lists that I create in GameHUb so the server can reference all the active and passive games currently in progress.

 

Signal R – Front End

Now we will make the front end that links with Signal R.

First we want to create a new blank Universal App.

 

 

Next we install the SignalR Client NuGet package for both the phone and the windows 8.1 project.

 

Now the way that the client interacts with the server code that was written is by connecting with the hub and sending messages through that hub.

For the SignalR portion of the client code create a new folder call SignalRCommunication, that contains the following classes: ISignalRHubs.cs, SignalREventArgs.cs, SignalRMessagingContainers.cs, and SignalRMessagingHub.cs.

The ISignalRHub is the interface for your SignalRMessagingHub.

 

And the SingnalREventArgs.cs acts as the interface allowing all parts of the project to access the messaging events.

The SignalRMessagingHub is where the connection is created between the server hub and client is initiated.

Now to connect with the Server Hub we need the following code:

#region “Implementation”

 

public async virtual void UserLogin(User tabletChatClient)

{

// Fire up SignalR Connection & join chatroom.

try

{

await gameConnection.Start();

 

if (gameConnection.State == Microsoft.AspNet.SignalR.Client.ConnectionState.Connected)

{

await SignalRGameHub.Invoke(“Login”, tabletChatClient);

}

}

catch (Exception ex)

{

 

// Do some error handling. Could not connect to Sever Error.

Debug.WriteLine(“Error: “+ ex.Message);

}

I put mine inside of the first User interaction with the server, so in this case when the User logs in.

For the Phone application if using the emulator you will need to use long polling since the phone emulator uses the PC’s identification number and a lot more work has to go into formatting your computer to run it.

This is the code you will need

#if WINDOWS_PHONE_APP

connection.Start(new LongPollingTransport());

#else

connection.Start();

#endif

 

Now we need to write code that will listen to events on the SignalR Server and wire them up appropriately

Calling the .On method is how our proxy hub listens to any messages the server passes to the clients.

Lets make this easier to understand by going through .On and explaining what each part is doing.

SignalRGameHub.On<Game, ServerMessage>(“gameCreated”, (g, sm) =>

{

SignalREventArgs gArgs = new SignalREventArgs();

gArgs.CustomGameObject = g;

gArgs.CustomServerMessage = sm;

// Raise custom event & let it bubble up.

SignalRServerNotification(this, gArgs);

});

The .On method can take in as many parameters as needed depending on which server call it’s defining On<x,y,z>.

Before you saw the server gamehub call Clients.All.gameCreated(); in the client code the quotes in SignalRGameHub.On method refer to which ever method we are listening to and then dictates what the client code should do based on the delegate following the On method.

The (g, sm) are the parameters that we defined earlier <x,y> in this case <Game, ServerMessage>. They are a part of a delegate that will create the SignalREventArgs, to parse out a gameObject and a ServerMessage. Then it raises a SignalRServerNotification(this, gArgs) that will trigger an event. You still need to write that event in other parts of your code.

We now need to write the method for the SignalRServerNotification(this, gArgs)

#region “Methods”

 

public virtual void OnSignalRServerNotificationReceived(SignalREventArgs e)

{

if (SignalRServerNotification != null)

{

SignalRServerNotification(this, e);

}

}

 

#endregion

 

 

 

Cool now on to defining calls made by the client that will be sent to the Server.

Make sure to define all the methods that will be sending information to the server. These async virtual methods will be called by the ISignalRHub. The quoted part “UpdateUser”, “CreateGame” and “JoinGame” refer to the methods on the server side GameHub, if the names are not exactly correct the server methods won’t invoke.

 

SingnalRMesagingContainers refers to the objects that you want to send through JSON to the server to manipulate. Meaning you would define your object classes within this .cs file. However if you already defined your models, in a models(or other name) folder, that will work too, in fact it’s preferred.

Referencing Gamehub

In order for your Signal R Hub to be accessed by all parts/pages of your project you will need to modify the App.xaml.cs in Shared.

public static new App Current

{

get { return Application.Current as App; }

}

Then inside the OnLaunched(LaunchActivatedEventArgs e) method add

App.Current.SignalRHub = new SignalRMessagingHub();

 

 

Inportant! If you are going to be passing objects through SignalR both projects need to have the exact same code for the objects.

Now when calling/listening to the SignalR hub from different pages of your app easy now that we have everything set up.

CALLING

LISTENING

Inorder to approipately handle the triggered SignalRServerNotifaction we need to add SignalRServerNorification for that page by referencing the App.Current.SignalRHub.SignalRServerNotification

When implementing the SignalRHub_SignalRServerNotification be sure to appropriately handle the dispatcher so the page can still be responsive when events are triggered.

 

In my code I use the CustomSererMessage to find which state the server is on versus the game. You can implement changing and checking game state however you think best suits your game.

 

Publishing your Server code

Eventually you want to publish your code to an azure website so anyone who downloads the application can connect to the server.

Use the steps in this tutorial to do so: http://www.asp.net/signalr/overview/deployment/using-signalr-with-azure-web-sites

Sphero

Okay so we finally have the backend set up. Now it’s time to implement more of the front end. Depending on your game you might want to change some of the XAML and front end to suit your purposes but the connection and control of the Sphero will remain the same.

First thing is to add the Sphero SDK that you downloaded to your project references.

https://github.com/SoatExperts/sphero-sdk

Connecting to Sphero

Second is to modify the App.xaml.cs so all pages in the project can remain connected to the same Sphero.

 

Now go into the page that you want to initiate connection with your sphero. For Hide and Seek I put it in the lobbyPage.xaml.cs since I wanted to make sure users were connected before they started the game. One thing to note about the DiscoverSpheros method; it will only return a list of Spheros that are on and in Bluetooth connection state (blinking red and blue).

private async void DiscoverSpheros()

{

try

{

// Discover paired Spheros

List<SpheroInformation> spheroInformations = new List<SpheroInformation>(await SpheroConnectionProvider.DiscoverSpheros());

 

if (spheroInformations != null && spheroInformations.Count > 0)

{

// Populate list with Discovered Spheros

SpherosDiscovered.ItemsSource = spheroInformations;

}

else

{

// No sphero Paired

MessageDialog dialogNSP = new MessageDialog(“No sphero Paired”);

await dialogNSP.ShowAsync();

}

 

}

catch (NoSpheroFoundException)

{

MessageDialog dialogNSF = new MessageDialog(“No sphero Found”);

dialogNSF.ShowAsync();

}

catch (BluetoothDeactivatedException)

{

// Bluetooth deactivated

MessageDialog dialogBD = new MessageDialog(“Bluetooth deactivated”);

dialogBD.ShowAsync();

}

}

After you discover the Spheros you then need to connect to one.

 

Controlling Sphero

With your device now connected to the Sphero that you choose, it’s time to implement the controls. Go to the page that you want the controls to appear and open the page’s .xaml file. In this case it was my GamePlayPage.xaml.

Then add the following to the <Page> tag properties:  xmlns:Controls=”using:Sphero.Controls”

Now depending on what else you want to put on the page the placement of this next code block will differ, but the content is still the same.

<Controls:Joystick x:Name=”spheroJoystick” HorizontalAlignment=”Left” Margin=”30,0,0,30″ Grid.Row=”1″ VerticalAlignment=”Bottom” Calibrating=”spheroJoystick_Calibrating” CalibrationReleased=”SpheroJoystick_CalibrationReleased” Moving=”SpheroJoystick_Moving” Released=”SpheroJoystick_Released” PointerReleased=”SpheroJoystick_PointerReleased”/>

The important properties of the Controls:Joystick input are Calibrating, CalibrationReleased, Moving, Released, and PointerReleased. These can be added directly in the XAML or can be added in the Events tab of the Properties Window in Visual Studio.

 

Almost done, with Sphero. In order for the code in XAML to actually do anything, we need to make sure that the events are implemented and make sure the connection is still active.

private SpheroDevice _spheroDevice;

We check the connection and start the Joystick in the page’s initialize method.

 

Since we saved the connection in the App.CurrentConnection we can reference even if we navigate to a different page after the initial connection.

 

And that’s it! You can now get more information from the Sphero if you want to, and track more data. Look in the API for on how to do more with your Sphero.

 

 

 

 

 

 

Drone

So some quick background on the Drone. ARDrones produce their own wifi that your device will need to connect to. Normally not a problem. But when your app needs to also talk to a Server consistently it provides a problem at we were only able to fix by connecting a PC to a hardline and the drones wifi simultaneously.

Getting Started

Create a new Blank windows 8.1 app. Download the SDK off git. Unzip the package and copy the reference libraries you need into your project Folder. Then go to your Reference folder in your project and reference the new files that you just moved. Do this in order to keep all the references together so when you share the solution nothing becomes dereferenced.

Integrating Front end

Go to the page’s xaml page and add to the <Page properties/>

xmlns:input=”using:ARDrone2Client.Windows.Input”

 

The main thing that needs to be in the UI is the field of view from the drone’s camera. The allure of drones is being able to control a new perspective that we normally can’t see as a human.  The camera output is also a major way that the user can control the drone when they lose direct line of sight.

To add this camera output to the screen you need to insert a MediaElement into your page. I like the element to take up almost the entire screen and then over lay the Joystick, Takeoff, Landing, and other command buttons.

 

The next step is to add the JoystickControl from the Drone ToolBox and the corresponding XAML Controls.

 

Be sure to name them differently. I suggest naming them after what they control. “YawGazJostick” and “RollPitchJoystick”.

 

 

<Grid x:Name=”JoystickGrid”

Grid.Row=”1″>

<input:JoystickControl x:Name=”YawGazJoystick”

BorderElipseColor=”Red”

StickColor=”Black”

Margin=”0,0,80,80″

HorizontalAlignment=”Right”

VerticalAlignment=”Bottom”

Height=”200″

Width=”200″ />

<input:JoystickControl x:Name=”RollPitchJoystick”

BorderElipseColor=”Green”

StickColor=”Black”

Margin=”80,0,0,80″

HorizontalAlignment=”Left”

VerticalAlignment=”Bottom”

Height=”200″

Width=”200″ />

</Grid>

 

It’s also good to have a Take Off button as well as an Emergency Button. There are several other capabilities of the Drone that you can add to your project however we won’t be using them. Your button doesn’t need to look like this it can be just a normal button.

 

 

<Button x:Name=”TakeOffLandButton”

Grid.Row=”1″

VerticalAlignment=”Bottom”

HorizontalAlignment=”Center”

IsEnabled=”False”

BorderThickness=”0″

Click=”TakeOffLandButton_Click”

Style=”{StaticResource ImageButtonStyle}”>

<!–<Image x:Name=”TakeOffLandImage”

Source=”/Assets/Parrot/btn_take_off_normal.png”

Stretch=”None” />–>

<Grid>

<Grid.ColumnDefinitions>

<ColumnDefinition Width=”50″ />

<ColumnDefinition Width=”*” />

<ColumnDefinition Width=”50″ />

</Grid.ColumnDefinitions>

<Polygon Grid.Column=”0″

Fill=”{StaticResource Green}”

StrokeThickness=”0″

Points=”0,62 50,0 50,62 0,62″ />

<Rectangle Grid.Column=”1″

Fill=”{StaticResource Green}”

HorizontalAlignment=”Stretch”

VerticalAlignment=”Stretch”

StrokeThickness=”0″ />

<Rectangle Fill=”{StaticResource ParrotBrush}”

Grid.Column=”1″

Height=”3″

VerticalAlignment=”Top” />

<Polygon  Grid.Column=”2″

Fill=”{StaticResource Green}”

StrokeThickness=”0″

Points=”0,0 50,62 0,62 0,0″ />

<Line Grid.Column=”0″

X1=”50″

Y1=”0″

X2=”1″

Y2=”60″

Stroke=”{StaticResource ParrotBrush}”

StrokeThickness=”3″ />

<Line Grid.Column=”2″

X1=”0″

Y1=”0″

X2=”50″

Y2=”60″

Stroke=”{StaticResource ParrotBrush}”

StrokeThickness=”3″ />

<TextBlock x:Name=”TakeOffLandingTextBlock”

Grid.Column=”1″

TextAlignment=”Center”

VerticalAlignment=”Center”

Foreground=”White”

FontSize=”30″

Padding=”50,0″>TAKE OFF</TextBlock>

</Grid>

</Button>

 

Same with the Emergency Button. It doesn’t need to look as complicated as this. Just make sure you include a click event.

<Button x:Name=”EmergencyButton”

Grid.Row=”0″

Grid.RowSpan=”2″

HorizontalAlignment=”Center”

VerticalAlignment=”Top”

BorderThickness=”0″

Padding=”0″

Margin=”50″

Width=”300″

Height=”60″

Click=”Emergency_Click”

VerticalContentAlignment=”Stretch”

HorizontalContentAlignment=”Stretch”

Style=”{StaticResource ImageButtonStyle}”>

<!–<Image Source=”/Assets/Parrot/btn_emergency_normal.png”

Stretch=”None” />–>

<Grid>

<Grid.ColumnDefinitions>

<ColumnDefinition Width=”50″ />

<ColumnDefinition Width=”*” />

<ColumnDefinition Width=”50″ />

</Grid.ColumnDefinitions>

<Polygon Grid.Column=”0″

Fill=”{StaticResource Red}”

StrokeThickness=”0″

Points=”1,0 50,0 50,60 1,0″ />

<Rectangle Grid.Column=”1″

Fill=”{StaticResource Red}”

HorizontalAlignment=”Stretch”

VerticalAlignment=”Stretch”

StrokeThickness=”0″ />

<Rectangle Fill=”{StaticResource ParrotBrush}”

Grid.Column=”1″

Height=”3″

VerticalAlignment=”Bottom” />

<Polygon  Grid.Column=”2″

Fill=”{StaticResource Red}”

StrokeThickness=”0″

Points=”0,0 50,0 0,60 0,0″ />

<Line Grid.Column=”0″

X1=”1″

Y1=”-1″

X2=”50″

Y2=”60″

Stroke=”{StaticResource ParrotBrush}”

StrokeThickness=”3″ />

<Line Grid.Column=”2″

X1=”0″

Y1=”60″

X2=”50″

Y2=”-2″

Stroke=”{StaticResource ParrotBrush}”

StrokeThickness=”3″ />

<TextBlock Grid.Column=”1″

TextAlignment=”Center”

VerticalAlignment=”Center”

Foreground=”White”

FontSize=”30″>EMERGENCY</TextBlock>

</Grid>

</Button>

 

 

 

 

Drone Backend .xaml.cs

Now in the xaml.cs page we need to initialize the connection to the Drone and establish eventhandlers for the joysticks inputs and the emergency land and take off buttons.

The page needs the appropriate library references:

using ARDrone2Client.Common;

using ARDrone2Client.Common.Configuration;

using ARDrone2Client.Common.Input;

Inside of the class for the page initialize the DroneClienct and VideoSourceUrl as well as the DispatcherTimer, which will allow the app to continue other processes while it connects to your Drone.

private readonly DroneClient _droneClient;

private const string VideoSourceUrl = “ardrone://192.168.1.1”;

// ReSharper disable once PrivateFieldCanBeConvertedToLocalVariable

private readonly DispatcherTimer _timer;

Add the lines below so the Drone can be initialized and connect to the application.

 

For the Drone portion of the application we also need to add lines to the OnNavigatedTo portion of the page.

if (!_droneClient.IsActive)

await _droneClient.ConnectAsync();

UpdateDisplay();

 

The UpdateDisplay() Method is in charge of changing the Screen to allow the other Drone Activities to become active. However we aren’t going to use many of them so the only one we need to become actuve is TakeOffLanding, EmergencyButton, and ResetEmergency Button.

 

Now to see the video while we are controlling the Drone we need to set the MediaElement’s source to the Video URL in the Page_OnLoaded method.

private void Page_OnLoaded(object sender, RoutedEventArgs e)

{

ArDroneMediaElem.Source = new Uri(VideoSourceUrl);

UpdateDisplay();

}

 

We also need to handle what happens to the video when the app suspends and resumes.

private void OnResume(object sender, object e)

{

ArDroneMediaElem.Source = new Uri(VideoSourceUrl);

}

 

private async void OnSuspend(object sender, Windows.ApplicationModel.SuspendingEventArgs e)

{

var deferral = e.SuspendingOperation.GetDeferral();

ArDroneMediaElem.Source = null;

await Task.Delay(2000);

deferral.Complete();

}

 

Handling Drone input and Joystick controls are similar to how we did with the Sphero.

OnTakeOffButton_Clicked

private void TakeOffLandButton_Click(object sender, RoutedEventArgs e)

{

if (_droneClient.IsFlying())

_droneClient.Land();

else

_droneClient.TakeOff();

UpdateDisplay();

}

Emergency

OnPointerEntered

base.OnPointerEntered(e);

 

if (e.Pointer.PointerDeviceType != Windows.Devices.Input.PointerDeviceType.Touch)

return;

 

Point pos = e.GetCurrentPoint(JoystickGrid).Position;

var joy = pos.X < (ActualWidth / 2) ? RollPitchJoystick : YawGazJoystick;

var bounds = new Rect(joy.ActualWidth / 2, joy.ActualHeight / 2, JoystickGrid.ActualWidth – joy.ActualWidth / 2, JoystickGrid.ActualHeight – 75 – joy.ActualHeight / 2);

if (!bounds.Contains(pos))

return;

joy.Margin = new Thickness(pos.X – joy.ActualWidth / 2, pos.Y – joy.ActualHeight / 2, 0, 0);

joy.VerticalAlignment = VerticalAlignment.Top;

joy.HorizontalAlignment = HorizontalAlignment.Left;

 

OnPointerReleased

base.OnPointerReleased(e);

if (e.Pointer.PointerDeviceType != Windows.Devices.Input.PointerDeviceType.Touch)

return;

Point pos = e.GetCurrentPoint(JoystickGrid).Position;

var joy = pos.X < (ActualWidth / 2) ? RollPitchJoystick : YawGazJoystick;

joy.Margin = joy == YawGazJoystick ? new Thickness(0, 0, 80, 80) : new Thickness(80, 0, 0, 80);

joy.VerticalAlignment = VerticalAlignment.Bottom;

joy.HorizontalAlignment = joy == YawGazJoystick ? HorizontalAlignment.Right : HorizontalAlignment.Left;

joy.Reset();

 

 

Summary

So like I said, not an easy endeavor. Working with hardware takes a while and comes with its own set of unforeseen challenges. But, I hope this little post helps.

If you want to look at the source code it’s here:

Though I do have a caveat, most of this code was written last year and has not been updated since 2014. If major changes have happened in the SDK they have not been integrated yet. I’ll try and keep an eye out and update accordingly.

 

Happy Coding!

-TheNappingKat

 

 


Leave a Reply

Your email address will not be published. Required fields are marked *