codehunt-logo

Designer Manual

This document describes the syntax and best techniques to create levels in Code Hunt.

Getting started

To access the level designer, sign in, click on [SETTINGS] then click on [DESIGNER]. In Office Mix, the designer is also used to author single levels.

The idea is to allow to define all the levels in the Code Hunt editor itself by extending C# or Java with custom macros (e.g. #level).

Below is an example of a single level authored in the designer.

#level Upside down
#code
public class Program {
public static int Puzzle(int x) {
return 0;
}
}
#secret
using System;
public static class Program {
public static int Puzzle(int x) {
PexAssume.IsTrue(x >= 0);
return -x;
}
}

where

  • #zone defines a new zone called “My Zone”
  • #sector starts a new sector called “Easy”.
  • #level starts a new level, “Good luck!” is the hint shown to the user.
  • #code defines a user code fragment in C#. That's the code shown when starting a level.
  • #secret defines the secret code fragment in C#. That's the code that the user needs to recover. Only C# is supported at this time.
  • PexAssume.IsTrue(x >= 0); retricts x to posivite numbers.
  • return x; is the secret code that players will have to discover.

You can also edit an entire zone with multiple sectors and levels by using the #zone and #sector macros.

#zone Elements
#sector Water
#level
...
#level
...
#sector Earth
#level
...

When editing a zone in Code Hunt, you can run the current level at any point. Code Hunt uses the caret position to find the closest level.

Authoring code fragments

In this section, you will learn how to tune your levels to improve the user experience.

How does the test generation work?

Code Hunt uses an automated test generation tool, Pex, to generate the table of inputs and outputs.

Pex finds interesting input values by analyzing the behavior of the code, combining dynamic and static analysis, and using a constraint solver. Pex seeks to generate a test suite with high code coverage, where each test case invokes the code with particular values.

When a user submits a code fragment, Code Hunt combines it with the secret fragment to create a testable program that Pex can analyze. The pseudo-code of the combined program looks as follows:

public static class Combined {
public static int Puzzle(int x) {
int ux = User.Program.Puzzle(x);
int sx = Secret.Program.Puzzle(x);
if (ux != sx)
throw new Exception("Mismatch!");
}
}

When Pex explores the combined program, it will try to generate an input such that ux != sx; in such case, Pex just found a failing test case where the user fragment did not match the secret fragment.

using System;
public static class Program {
public static bool Puzzle(int[] a) {
if (a != null &&
a.Length > 0 &&
a[0] == 1234567890)
return true;
else
return false;
}
}

Pex performs the following process in an iterative fashion.

  • Pex starts by choosing some inputs. Pex always begins with the simplest inputs possible. In the above example, the input would be null for the array parameter.
  • Then Pex executes the code, and monitors all conditions that are checked, along the execution path that is taken for the chosen inputs. For the example above, the method returns because a == null.
  • Pex negates this condition, i.e. a != null, and queries a constraint solver to determine whether there is a solution for the negated condition.
  • If a solution exists, then this solution represents another test input, which would cause the code to take a different path.

And then Pex repeats this process. Internally, Pex represents all conditions that the program checks as a tree.

dsetree


Figure 1. The dynamic exploraiton tree

Every time Pex executes the code again, the execution tree might grow, as Pex learns about new behaviors of the code.

After a few iterations, Pex finishes for this example.

dsetable


Figure 2. The table of constraints

When the tree grows big, Pex can choose during each iteration of the process which direction to extend the tree. For this example, this choice is always simple, because the code has no complicated branching structure, and no loops.

Input types

The signature of your Puzzle main contains any number of int, byte, bool, string arguments; arrays and jagged arrays of those. Avoid using floating point numbers (float or double) as the constraint solver in Pex has limitation dealing with those types.

using System;
using Microsoft.Pex.Framework;
public class Program {
public static int[] Puzzle(int x, byte b, bool v, string s, int[] a) {
...
}
}

Custom types can be used internally but cannot be exposed on the Puzzle signature.

Restricting inputs PexAssume

As a test generation engine, Pex tries to generate the smallest test case possible.nHowever, it may decide to use values that make it harder to reason: large numbers, weird unicode characters etc

Let's take a look at a simple code fragment:

using System;
using Microsoft.Pex.Framework;
public class Program {
public static int Puzzle(int x) {
...
}
}

You could make the duel a little easier by restricting x to positive numbers. To do so, we are going to add PexAssume.IsTrue(x >= 0) at the start of Puzzle:

using System;
using Microsoft.Pex.Framework;
public class Program {
public static int Puzzle(int x) {
PexAssume.IsTrue(x >= 0);
...
}
}

Of course, you can use arbitrary expressions in PexAssume.IsTrue(). For example, we can also make x less than 1000:

using System;
using Microsoft.Pex.Framework;
public class Program {
public static int Puzzle(int x) {
PexAssume.IsTrue(x >= 0);
PexAssume.IsTrue(x < 1000);
...
}
}

Overflows

Integer arithmetic may overflow and lead to surprising results! For example, in math class you would learn that there is no natural number x such that x > x + 10. In a computer natural numbers are not infinite and eventually, this equation will be true (for x = 2147483640)! You can use assumptions to avoid overflows.

Here are other typical PexAssume functions that you will find useful in your code fragments:

PexAssume.IsNotNull(Object value): avoids a null value; very useful with arrays and strings

using System;
using Microsoft.Pex.Framework;
public class Program {
public static int Puzzle(int[] a) {
PexAssume.IsNotNull(a);
...
}
}

PexAssume.IsNotNullOrEmpty(string value): avoids a null or empty string;

using System;
using Microsoft.Pex.Framework;
public class Program {
public static int Puzzle(int[] a) {
PexAssume.IsNotNullOrEmpty(a);
...
}
}

Restricting inputs Advanced

It is often more efficient to build a single boolean expression and feed it to Pex is a single PexAssume.IsTrue call. To avoid creating branches, use the logical and (&) instead of the conditional and (&&).

using System;
using Microsoft.Pex.Framework;
public class Program {
public static int Puzzle(int x) {
PexAssume.IsTrue(x > 0 & x < 1000);
...
}
}

When evaluting assumption for a string or array, accumulate the boolean expression in a local. For example, the following assumption ensures that all numbers in a are positive.

using System;
using Microsoft.Pex.Framework;
public class Program {
public static int Puzzle(int[] a) {
bool b = true;
for(int i = 0; i < a.Length; ++i)
b &= a[i] > 0;
PexAssume.IsTrue(b);
...
}
}

Mutating inputs

Never mutate input arrays in-place. Instead, clone the array first; then apply any transformation.

// DO NOT DO THIS
public static int[] Puzzle(int[] a) {
Array.Sort(a); // Don't do this. Never mutate an input.
return a;
}
// Clone and mutate...
public static int[] Puzzle(int[] a) {
int[] b = (int[])Array.Clone(a); // clone a into b
Array.Sort(b); // in-place sort of b
return b;
}

Scafolding levels (UNDER CONSTRUCTION)

You can request Code Hunt to use the user code from the previous level as a starting code for the user. This allows to scafold levels of increasing complexity. Scafolding is enabled by enabled the #levelUsePreviousCode macro under the #level section.

#level
#levelUsePreviousCode
#secret
...

When scafolding is enabled, Code Hunt will expand the macro the first time the level is accessed by the user. Further modification will not be tracked.

You can still provide a #code macro for testing purposes.

#level
#levelUsePreviousCode
#code
...
#secret
...

Other random collection of tips

Prefer for loops to foreach loops: there is a tiny overhead associated to foreach.

// better this
for(int i = 0; i < a.Length; ++i)
... a[i] ...
// than this
foreach(var ai in a)
... ai ...

Use empty if statements to force initial test cases

public static int Puzzle(int[] a) {
if (x == 42 & y == -42); // force (x=42, y=-42) pair
...

All arithmetic operations are unchecked by default - so (confusing) overflows may happen. In C#, use the checked statement to force a checked context:

public static int Puzzle(int[] a) {
if (x == 42 & y == -42); // force (x=42, y=-42) pair
...

Office Mix

You can insert Code Hunt levels in Office Mix presentations to create interactive coding lessons. Try this online lesson to see it in action.

The source is automatically saved in the slide when you hit the Capture button.

If you share your slidedeck, only you will be able to access the source of the secret program. Code Hunt automatically replace the secret section with a #levelSecretId macro that references the secret program. Thus, other users won't be able to see your secret program!

#level Good luck!
#levelSecretId abc...
#code
using System;
public class Program {
public static int Puzzle(int x) {
return 0;
}
}

Macros

This section lists all the macros supported by the Code Hunt syntax.

#zone <friendly name>

Defines a new zone named <friendly name>. This is the first macro that needs to appear in your file. May be followed by multiple sector definitions.

The following optional macros can be used to customize a zone. They must be used before any #sector definition.

#zoneInfoUrl <url> optional

Specifies a <url> pointing to the information about this zone. URL must be using http or https protocols.

#zoneLogoSplashUrl <url> optional

Specifies a <url> pointing to the picture logo to be displayed on the landing page, above the ‘Code Hunt’ splash picture. The picture must have a transparent background, be of minimum size 225x120 (larger pictures should use the same aspect ratio).

#zoneLogoMenuUrl <url> optional

Specifies a <url> pointing to the picture logo to be displayed on the menu next to the white ‘Code Hunt’ icon. The picture must have a transparent background, of size 40x22. Preferrably, this logo should use a white theme to match the rest of the icons.

#zoneNoHints optional

Disable generating hints for users.

#zoneOffsetRating <0..100> optional

Constant number to be added to level rating before computing final score. Default is 0, max 100.

#zoneStartTime <UTC date time> optional

The starting UTC date time when this universe is active. Users will not be able to capture code before that time.

The date time must be a UTC date time by appending a ‘Z’ (2002-05-30T09:30:10Z) or by specifying an negative or positive offset behind the time (2002-05-30T09:30:10-06:00).

#zoneEndTime <UTC date time> optional

The end UTC date time when this universe is active. Users will not be able to capture code after that time.

The date time must be a UTC date time by appending a ‘Z’ (2002-05-30T09:30:10Z) or by specifying an negative or positive offset behind the time (2002-05-30T09:30:10-06:00).

#zoneRequireUserId

Requires that all players using this zone enter a user id. The default value is false. A user id can automatically be provisionned by a URL-based signin. Otherwise, the user will have to manually enter the code through the settings.

#sector <friendly name>

Starts the definition of a new sector named friendly name. May be followed by multiple level definitions (up to 15).

#sectorScoreWeigth <1..100> optional

This factor is used to multiple the score of the current sector. Default is 1.

#level <description>

Starts the definition of a level. May be following by 1 or more user code fragments and 1 secret code fragment. The description is optional.

#levelSecretDescription <description> optional

A description of the secret code. Used by the zone designers only and not visible to users.

#levelSecretId <secretid> optional

Instead of providing the full source of the secret program, specify the program id of the secret source code. This macro allows to define levels without having access to the source of a secret program. When using this macro, you should not use the #secret macro.

#levelUsePreviousCode optional

Specifies that this level scafolds from the previous level. This macro cannot be used on the first level of a sector.

#code <|C#|Java>

Starts a user code fragment of the given language (C# or Java). If omitted, the language is assumed to be C#. This is the code fragment that the user starts from. The fragment runs until the next macro.

#secret <|C#>

Starts a secret code fragment. This is the fragment that the user needs to capture. C# is the only language currently supported. The fragment runs until the next macro. This