When I was looking at Javascript frameworks like Angular and ReactJS I kept running into these weird reducers that were used in state management mostly. It all felt so unnecessarily complicated, so I didn't look too closely into it. Today, reading some random post on dev.to, I found this simple and concise piece of code that explains it:

// simple to unit test this reducer
function maximum(max, num) { return Math.max(max, num); }

// read as: 'reduce to a maximum' 
let numbers = [5, 10, 7, -1, 2, -8, -12];
let max = numbers.reduce(maximum);

Kudos to David for the code sample.

The reducer, in this case, is a function that can be fed to the the reduce function, which is known to developers in Javascript and a few other languages, but which for .NET developers it's foreign. In LINQ, we have Aggregate!

// simple to unit test this Aggregator ( :) )
Func<int, int, int> maximum = (max, num) => Math.Max(max, num);

// read as: 'reduce to a maximum' 
var numbers = new[] { 5, 10, 7, -1, 2, -8, -12 };
var max = numbers.Aggregate(maximum);

Of course, in C# Math.Max is already a reducer/Aggregator and can be used directly as a parameter to Aggregate.

I found a lot of situations where people used .reduce instead of a normal loop, which is why I almost never use Aggregate, but there are situations where this kind of syntax is very useful. One would be in functional programming or LINQ expressions that then get translated or optimized to something else before execution, like SQL code. (I don't know if Entity Framework translates Aggregate, though). Another would be where you have a bunch of reducers that can be used interchangeably.

Intro

  I want to examine together with you various types of sort algorithms and the tricks they use to lower the magic O number. I reach the conclusion that high performance algorithms that are labeled as specific to a certain type of data can be made generic or that the generic algorithms aren't really that generic either. I end up proposing a new form of function that can be fed to a sorting function in order to reach better performance than the classic O(n*log(n)). Extra bonus: finding distinct values in a list.

Sorting

  But first, what is sorting? Given a list of items that can be compared to one another as lower or higher, return the list in the order from lowest to highest. Since an item can be any type of data record, to define a generic sorting algorithm we need to feed it the rules that make an item lower than another and that is called the comparison function. Let's try an example in Javascript:

  // random function from start to end inclusive
  function rand(start, end) {
    return parseInt(start + Math.random() * (end - start + 1));
  }
  
  // measure time taken by an action and output it in console
  let perfKey = 0;
  function calcPerf(action) {
    const key = perfKey++;
    performance.mark('start_' + key);
    action();
    performance.mark('end_' + key);
    const measure = performance.measure('measure_' + key, 'start_' + key, 'end_' + key);
    console.log('Action took ' + measure.duration);
  }
  
  // change this based on how powerful the computer is
  const size = 10000000;
  // the input is a list of size 'size' containing random values from 1 to 50000
  const input = [];
  for (let i = 0; i < size; i++)
    input.push(rand(1, 50000));
  
  // a comparison function between two items a and b
  function comparisonFunction(a, b) {
    if (a > b)
      return 1;
    if (a < b)
      return -1;
    return 0;
  }
  
  const output = [];
  // copy input into output, then sort it using the comparison function
  // same copying method will be used for future code
  calcPerf(() => {
    for (let i = 0; i < size; i++)
      output.push(input[i]);
    output.sort(comparisonFunction);
  });

  It's not the crispest code in the world, but it's simple to understand:

  • calcPerf is computing the time it takes for an action to take and logs it to the console
  • start by creating a big array of random numbers as input
  • the array in a result array and sorting it with the default sort function, to which we give the comparison function
  • display the time it took for the operation.

  This takes about 4500 milliseconds on my computer.

  Focus on the comparison function. It takes two items and returns a number that is -1, 0 or 1 depending on whether the first item is smaller, equal or larger than the second. Now let's consider the sorting algorithm itself. How does it work?

  A naive way to do it would be to find the smallest item in the list, move it to the first position in the array, then continue the process with the rest of the array. This would have a complexity of O(n2). If you don't know what the O complexity is, don't worry, it just provides an easy to spell approximation of how the amount of work would increase with the number of items in the input. In this case, 10 million records, squared, would lead to 100 trillion operations! That's not good.

  Other algorithms are much better, bringing the complexity to O(n*log(n)), so assuming base 10, around 70 million operations. But how do they improve on this? Surely in order to sort all items you must compare them to each other. The explanation is that if a<b and b<c you do not need to compare a to c. And each algorithm tries to get to this in a different way.

  However, the basic logic of sorting remains the same: compare all items with a subset of the other items.

Partitioning

  A very common and recommended sorting algorithm is QuickSort. I am not going to go through the entire history of sorting algorithms and what they do, you can check that out yourself, but I can focus on the important innovation that QuickSort added: partitioning. The first step in the algorithm is to choose a value out of the list of items, which the algorithm hopes it's as close as possible to the median value and is called a pivot, then arrange the items in two partitions: the ones smaller than the pivot and the ones larger than the pivot. Then it proceeds on doing the same to each partition until the partitions are small enough to be sorted by some other sort algorithm, like insertion sort (used by Chrome by default).

  Let's try to do this manually in our code, just the very first run of the step, to see if it improves the execution time. Lucky for us, we know that the median is around 25000, as the input we generated contains random numbers from 1 to 50000. So let's copy the values from input into two output arrays, then sort each of them. The sorted result would be reading from the first array, then from the second!

  // two output arrays, one for numbers below 25000, the other for the rest
  const output1 = [];
  const output2 = [];
  const pivot = 25000;
  
  calcPerf(() => {
    for (let i = 0; i < size; i++) {
      const val = input[i];
      if (comparisonFunction(val, pivot) < 0)
        output1.push(val);
      else
        output2.push(val);
    }
    // sorting smaller arrays is cheaper
    output1.sort(comparisonFunction);
    output2.sort(comparisonFunction);
  });

  Now, the performance is slightly better. If we do this several times, the time taken would get even lower. The partitioning of the array by an operation that is essentially O(n) (we just go once through the entire input array) reduces the comparisons that will be made in each partition. If we would use the naive sorting, partitioning would reduce nto n+(n/2)2+(n/2)2 (once for each partitioned half), thus n+n2/2. Each partitioning almost halves the number of operations!

  So, how many times can we half the number of operations for? Imagine that we do this with an array of distinct values, from 1 to 10 million. In the end, we would get to partitions of just one element and that means we did a log2(n) number of operations and for each we added one n (the partitioning operation). That means that the total number of operations is... n*log(n). Each algorithm gets to this in a different way, but at the core of it there is some sort of partitioning, that b value that makes comparing a and c unnecessary.

  Note that we treated the sort algorithm as "generic", meaning we fed it a comparison function between any two items, as if we didn't know how to compare numbers. That means we could have used any type of data as long as we knew the rule for comparison between items.

  There are other types of sorting algorithms that only work on specific types of data, though. Some of them claim a complexity of O(n)! But before we get to them, let's make a short detour.

Distinct values

  Another useful operation with lists of items is finding the list of distinct items. From [1,2,2,3] we want to get [1,2,3]. To do this, we often use something called a trie, a tree-like data structure that is used for quickly finding if a value exists or not in a list. It's the thing used for autocorrect or finding a word in a dictionary. It has an O(log n) complexity in checking if an item exists. So in a list of 10 million items, it would take maybe 20 operations to find the item exists or not. That's amazing! You can see that what it does is partition the list down to the item level.

  Unfortunately, this only works for numbers and strings and such primitive values. If we want to make it generic, we need to use a function that determines when two items are equal and then we use it to compare to all the other items we found as distinct so far. That makes using a trie impossible.

  Let me give you an example: we take [1,1,2,3,3,4,5] and we use an externally provided equality function:

  • create an empty output of distinct items
  • take first item (1) and compare with existing distinct items (none)
  • item is not found, so we add it to output
  • take next item (1) and compare with existing distinct items (1)
  • item is found, so we do nothing
  • ...
  • we take the last item (5) and compare with existing items (1,2,3,4)
  • item is not found, so we add it to the output

  The number of operations that must be taken is the number of total items multiplied by the average number of distinct items. That means that for a list of already distinct values, the complexity if O(n2). Not good! It increases exponentially with the number of items. And we cannot use a trie unless we have some function that would provide us with a distinctive primitive value for an item. So instead of an equality function, a hashing function that would return a number or maybe a string.

  However, given the knowledge we have so far, we can reduce the complexity of finding distinct items to O(n*log(n))! It's as simple as sorting the items, then going through the list and sending to output an item when different from the one before. One little problem here: we need a comparison function for sorting, not an equality one.

So far

  We looked into the basic operations of sorting and finding distinct values. To be generic, one has to be provided with a comparison function, the other with an equality function. However, if we would have a comparison function available, finding distinct generic items would become significantly less complex by using sorting. Sorting is better than exponential comparison because it uses partitioning as an optimization trick.

Breaking the n*log(n) barrier

  As I said above, there are algorithms that claim a much better performance than n*log(n). One of them is called RadixSort. BurstSort is a version of it, optimized for strings. CountSort is a similar algorithm, as well. The only problem with Radix type algorithms is that they only work on numbers or recursively on series of numbers. How do they do that? Well, since we know we have numbers to sort, we can use math to partition the lot of them, thus reducing the cost of the partitioning phase.

  Let's look at our starting code. We know that we have numbers from 1 to 50000. We can find that out easily by going once through all of them and computing the minimum and maximum value. O(n). We can then partition the numbers by their value. BurstSort starts with a number of "buckets" or lists, then assigns numbers to the buckets based on their value (dividing the value to the number of buckets). If a bucket becomes too large, it is "burst" into another number of smaller buckets. In our case, we can use CountSort, which simply counts each occurrence of a value in an ordered array. Let's see some code:

  const output = [];
  const buckets = [];
  calcPerf(() => {
    // for each possible value add a counter
    for (let i = 1; i <= 50000; i++)
      buckets.push(0);
    // count all values
    for (let i = 1; i <= size; i++) {
      const val = input[i];
      buckets[val - 1]++;
    }
    // create the output array of sorted values
    for (let i = 1; i <= 50000; i++) {
      const counter = buckets[i - 1];
      for (let j = 0; j < counter; j++)
        output.push(i);
    }
  });

  This does the following:

  • create an array from 1 to 50000 containing zeros
  • for each value in the input, increment the bucket for that value
  • at the end just go through all of the buckets and output the value as many times as the value in the bucket shows

  This algorithm generated a sorted output array in 160 milliseconds!

  And of course, it is too good to be true. We used a lot of a priori knowledge:

  • min/max values were already known
  • the values were conveniently close together integers so we can use them as array indexes

  I can already hear you sigh "Awwh, so I can't use it!". Do not despair yet!

  The Radix algorithm, that is used only for numbers, is also used on strings. How? Well, a string is reducible to a list of numbers (characters) so one can recursively assign each string into a bucket based on the character value at a certain index. Note that we don't have to go through the entire string, the first few letters are enough to partition the list in small enough lists that can be cheaply sorted.

  Do you see it yet?

A generic partition function

  What if we would not use an equality function or a comparison function or a hashing function as a parameter for our generic sort/distinct algorithm? What if we would use a partition function? This partition function would act like a multilevel hashing function returning values that can also be compared to each other. In other words, the generic partition function could look like this:

  function partitionFunction(item, level) returning a byte

  For strings it returns the numeric value of the character at position level or 0. For numbers it returns the high to low byte in the number. For object instances with multiple properties, it would return a byte for each level in each of the properties that we want to order by. Radix style buckets would use the known values from 0 to 255. The fact that the multilevel partitioning function is provided by the user means we can pack in it all the a priori knowledge we have, while keeping the sorting/distinct algorithm unchanged and thus, generic! The sorting will be called by providing two parameters: the partitioning function and the maximum level to which it should be called:

  sort(input, partitioningFunction, maxLevel)

A final example

  Here is an implementation of a radix sorting algorithm that receives a multilevel partitioning function using our original input. Note that it is written so that it is easily read and not for performance:

  // will return a sorted array from the input array
  // using the partitioning function up to maxLevel
  function radixSort(input, partitioningFunction, maxLevel) {
    let buckets = Array.from({length: 256}, () => []);
    buckets[0] = input;
    // reverse order, because level 0 should be the most significant
    for (let level = maxLevel-1; level >=0; level--) {
      let tempBuckets = Array.from({length: 256}, () => []);
      for (let bucketIndex = 0; bucketIndex < buckets.length; bucketIndex++) {
        const bucket = buckets[bucketIndex];
        const bucketLength = bucket.length;
        for (let bucketOffset = 0; bucketOffset < bucketLength; bucketOffset++) {
          const val = bucket[bucketOffset];
          const partByte = partitioningFunction(val, level);
          tempBuckets[partByte].push(val);
        }
      }
      buckets = tempBuckets;
    }
    const output = [].concat(...buckets);
    return output;
  }

  // return value bytes, from the most significant to the least
  // being <50000 the values are always 2 bytes  
  function partitioningFunction(item, level) {
    if (level === 0) return item >> 8;
    if (level === 1) return item & 255;
    return 0;
  }
  
  let output3 = [];
  calcPerf(() => {
    output3 = radixSort(input, partitioningFunction, 2);
  });

Want to know how long it took? 1300 milliseconds!

You can see how the same kind of logic can be used to find distinct values, without actually sorting, just by going through each byte from the partitioning function and using them as values in a trie, right?

Conclusion

Here is how a generic multilevel partitioning function replaces comparison, equality and hashing functions with a single concept that is then used to get high performance from common data operations such as sorting and finding distinct values.

I will want to work on formalizing this and publishing it as a library or something like that, but until then, what do you think?

Wait, there is more

There is a framework in which something similar is being used: SQL. It's the most common place where ORDER BY and DISTINCT are used. In SQL's case, we use an optimization method that uses indexes, which are also trie data structures storing the keys that we want to order or filter by. Gathering the data to fill a database index also has its complexity. In this case, we pre-partition once and we sort many. It's another way of reducing the cost of the partitioning

However, this is just a sub-type of the partition function that I am talking about, one that uses a precomputed data structure to reach its goal. The multilevel partition function concept I am describing here may be pure code or some other encoding of information we know out of hand before doing the operation.

Finally, the complexity. What is it? Well instead of O(n*log(n)) we get O(n*k), where k is the maximum level used in the partition function. This depends on the data, so it's not a constant, but it's the closest theoretical limit for sorting, closer to O(n) than the classic log version. I am not the best algorithm and data structure person, so if you have ideas about it and want to help me out, I would be grateful.  

  I am subscribed to the StackOverflow newsletter and most of the times the "top" questions there are really simple things that gain attention from a lot of people. Today I got one question that I would have thought has an obvious answer, but it did not.

  The question was what does "asdf".replace(/.*/g,"x") return? 

  And the answer to the question "What does a regular expression replace of everything with x return?" is.... [Ba da bum!] "xx".

  The technical answer is there in the StackOverflow question, but I am gonna walk you through some steps to get to understand this the... dumb way.

  So, let's try variations on the same theme. What does "asdf".matchAll(/.*/g) return? Well, first of all, in Chrome, it returns a RegExpStringIterator, which is pretty cool, because it's already using the latest Javascript features and it is returning an iterator rather than an array. But we can just use Array.from on it to get an array of all matches: for "asdf" and for "".

  That's a pretty clear giveaway. Since the regular expression is a global one, it will get a match, then the next one until there is nothing left. First match is "asdf" as expected, the next one is "", which is the rest of the string and which also matches .* Why is it, then, that it doesn't go into a stack overflow (no pun intended) and keep turning up empty strings? Again, it's an algorithm described in an RFC and you need a doctorate in computer science to read it. Well, it's not that complicated, but I did promise a dumb explanation.

   And that is that after you get a match on an index, the index is incremented. First match is found at index 0, the next one at 4. There are no matches from index 5 on.

   Other variations on this theme are "asdf".matchAll(/.?/g), which will return "a","s","d","f","". You can't do "asdf".matchAll(/.*/) , you get a TypeError: undefineds called with a non-global RegExp argument error that really doesn't say much, but you can do "asdf".match(/.*/g) which returns just an array of strings, rather than more complex objects. You can also do

var reg = /.*/g;
console.log(reg.exec("asdf"),reg.exec("asdf"),reg.exec("asdf"),reg.exec("asdf"))

This more classic approach will return "asdf", "", "", "" and it would continue to return empty strings ad infinitum!

But how should one write a regular expression to get what you wanted to get, a replacement of everything with x? /.+/g would work, but it would not match an empty string. On the other hand, when was the last time you wanted to replace empty strings with anything?

  I am going to do this in Javascript, because it's easier to write and easier for you to test (just press F12 in this page and write it in the console), but it applies to any programming language. The issue arises when you want to execute a background job (a setTimeout, an async method that you do not wait, a Task.Run, anything that runs on another execution path than the current one) inside a for loop. You have the index variable (from 0 to 10, for example) and you want to use it as a parameter for the background job. The result is not as expected, as all the background jobs use the same value for some reason.

  Let's see a bit of code:

// just write in the console numbers from 0 to 9
// but delayed for a second
for (var i=0; i<10; i++)
{
  setTimeout(function() { console.log(i); },1000);
}

// the result: 10 values of 10 after a second!

But why? The reason is the "scope" of the i variable. In this case, classic (EcmaScript 5) code that uses var generates a value that exists everywhere in the current scope, which for ES5 is defined as the function this code is running from or the global scope if executed directly. If after this loop we write console.log(i) we get 10, because the loop has incremented i, got it to 10 - which is not less than 10, and exited the loop. The variable is still available. That explains why, a second later, all the functions executed in setTimeout will display 10: that is the current value of the same variable.

Now, we can solve this by introducing a local scope inside the for. In ES5 it looked really cumbersome:

for (var i=0; i<10; i++)
{
  (function() {
    var li = i;
    setTimeout(function() { console.log(li); },1000);
  })();
}

The result is the expected one, values from 0 to 9.

What happened here? We added an anonymous function and executed it. This generates a function scope. Inside it, we added a li variable (local i) and then set executing the timeout using that variable. For each value from 0 to 9, another scope is created with another li variable. If after this code we write console.log(li) we get an error because li is undefined in this scope. It is a bit confusing, but there are 10 li variables in 10 different scopes.

Now, EcmaScript 6 wanted to align Javascript with other common use modern languages so they introduced local scope for variables by defining them differently. We can now use let and const to define variables that are either going to get modified or remain constant. They also exist only in the scope of an execution block (between curly brackets).

We can write the same code from above like this:

for (let i=0; i<10; i++) {
  const li = i;
  setTimeout(()=>console.log(li),1000);
}

In fact, this is more complex than it has to be, but that's because another Javascript quirk. We can simplify this for the same result as:

for (let i=0; i<10; i++) {
  setTimeout(()=>console.log(i),1000);
}

Why? Because we "let" the index variable, so it only exists in the context of the loop execution block, but apparently it creates one version of the variable for each loop run. Strangely enough, though, it doesn't work if we define it as "const".

Just as an aside, this is way less confusing with for...of loops because you can declare the item as const. Don't use "var", though, or you get the same problem we started with!

const arr=[1,2,3,4,5];
for (const item of arr) setTimeout(()=>console.log(item),1000);

In other languages, like C#, variables exist in the scope of their execution block by default, but using a for loop will not generate multiple versions of the same variable, so you need to define a local variable inside the loop to avoid this issue. Here is an example in C#:

for (var i=0; i<10; i++)
{
    var li = i;
    Task.Run(() => Console.WriteLine(li));
}
Thread.Sleep(1000);

Note that in the case above we added a Thread.Sleep to make sure the app doesn't close while the tasks are running and that the values of the loop will not necessarily be written in order, but that's beside the point here. Also, var is the way variables are defined in C# when the type can be inferred by the compiler, it is not the same as the one in Javascript.

I hope you now have a better understanding of variable scope.

Intro

  As I was working on LInQer I was hooked on the multiple optimizations that I was finding. Do you want to compute the average of an iterable? You would need the total count and the sum of the items, which you can get in a single function that you can reuse to get the sum or the count. But what if the iterable is an integer range between 1 and 10? Then you can compute the sum and you already know the count. Inspired by that work and by other concepts like interval types or Maybe/Nullable types, I've decided to write this post, which I do not know if it will lead to any usable code.

What is an iterable/enumerable?

  In Javascript they call it an Iterable, in .NET you have IEnumerable. They mean the same thing: sources of values. With new concepts like async/await you can use Observables as Enumerables as well, although they are theoretically diametrically opposing patterns. In both languages they are implemented as having a method that returns an iterator/enumerator, an object that can move to the next value, give you the next value and maybe reset itself. You can define any stream of values like that, having one or more values or, indeed, none. From now own I will discuss in terms of .NET nomenclature, but I see no reason why it wouldn't apply to any other language that implements this feature.

  For example an array is defined as an IEnumerable<T> in .NET. Its enumerator will return false if trying to move to the next value and the array is empty, or true if there is at least a value and the current value will return the first value in the array. Move next again and it will return true or false depending on whether there is a next value. But there is no need for the values to exist to have an Enumerable. One could write an object that would return all the positive integer numbers. It's length would be infinite and the values would only be generated when requested. There is no differentiation between an Enumerable and a generator in .NET, but there is in Javascript. For this reason whenever I will use the term generator, I will mean an object that generates values rather than produce them from a source of existing ones.

The NULL controversy

  A very popular InfoQ post describes the introduction of the NULL concept in programming languages a the billion dollar mistake. I am not so sure about that, but I can concede they make good points. The alternative to using a special value to describe the absence of a value is use an "option" object that either has Some value or it has None. You would check the existence of a value by calling a method to tell you if it has a value and you would get the value from the current value property. Doesn't it sound familiar? It's a more specific case of an Enumerator! Another popular solution to remove NULLs from code is to never return values from your methods, but arrays. An empty array would represent no value. An array is an Enumerable!

And that last idea opens up an interesting possibility: instead of one or none, you can have multiple values. What then? What would a multiplication mean? What about a decision block?

The LInQer experience

  If you know me, you are probably fed up with me plugging LInQer as the greatest thing since fire was invented. But that's because it is! And while implementing .NET LInQ as a Javascript library I've played with some very interesting concepts.

  For example, while implementing the Last operator on enumerables, I had two different implementations depending on whether one could know the length in advance and one could use indexed access to the values. An array of one billion values has no problem giving you the last item in it because of two things: you know where the array ends and you can access any item at any position without having to go through other values. You just take the value at index one billion minus one. If you would have a generator, though, the only way to get the last value would be to enumerate again and again and again and only when moving to the next value would fail you would have the last value as the last one. And of course, this would be bad if there are no bounds to the generator.

  But there is more. What about very common statistical values like the sum? This, of course, applies to numbers. The Enumerable need not produce numbers, so in other contexts it means nothing. Then there are concepts like statistical distribution. One can make some assumptions if they know the distribution of values. A constant yet infinite generator of numbers will always have the same average value. It would return the same value, regardless of index.

  I spent a lot of time doing sorting that only needs a part of the enumerable, or partial sorting. I've implemented a Quicksort algorithm that works faster than the default sort when there are enough values and that can ignore the parts of the array that I don't need. Also, there are specific algorithm to return the last or first N items. All of this depends on functions that determine the order of items. Randomness is also interesting, as it needs to take into consideration the change of probabilities as the list of items increases with each request. Sampling was fun, too!

  Then there were operators like Distinct or Group which needed to use functions to determine sameness.

  With all this work, I've never intended to make this more than what LInQ is in .NET: a way to dynamically filter and map and enumerate sequences of items without having to go through them all or to create intermediate but unnecessary arrays. What I am talking about now is taking things further and deeper.

Continuous intervals

  What if the Enumerable abstraction is not enough? For example one could define the interval of real numbers between 0 and 1. You can never enumerate the next value, but there are definite boundaries, a clear distribution of values, a very obvious average. What about series and limits? If a generator generates values that depend on previous values, like a geometric progression or a Fibonacci series, you can sometimes compute the minimum or maximum value of the items in it, or of their sums.  

Tools

  So we have more concepts in our bag now:

  • move next (function)
  • current value
  • item length (could be infinite or just unknown)
  • indexed access (or not)
  • boundaries (min, max, limits)
  • distribution (probabilities)
  • order
  • discreteness

  How could we use these?

Concrete cases

  There is one famous probabilities problem: what are the chances you will get a particular value by throwing a number of dice. And it is interesting because there is a marked difference between using one die or more. Using at least two dice and plotting the values you get after multiple throws you get what is called a Normal distribution, a Gauss curve, and that's because there are more combinations of values that sum up to 6 than there are for 2.

  How can we declare a value that belongs to an interval? One solution is to add all kinds of metadata or validations. But what if we just declare an iterable with one value that has a random value between 1 and 6? And what if we add it with another one? What would that mean?

  Here is a demo example. It's silly and it looks too much like the Calculator demos you see for unit testing and I really hate those, but I do want to just demo this. What else can we do with this idea? I will continue to think about it.

class Program
    {
        static void Main(string[] args)
        {
            var die1 = new RandomGenerator(1, 6);
            var die2 = new RandomGenerator(1, 6);
            // just get the value
            var value1 = die1.First() + die2.First();
            // compose the two dice using Linq, then get value
            var value2 = die1.Zip(die2).Select(z => z.First + z.Second).First();
            // compose the two dice using operator overload, then get value
            var value3 = (die1 + die2).First();
            var min = (die1 + die2).Min();
        }

        /// <summary>
        /// Implemented Min alone for demo purposes
        /// </summary>
        /// <typeparam name="T"></typeparam>
        public interface IGenerator<T> : IEnumerable<T>
        {
            int Min();
        }

        /// <summary>
        /// Generates integer values from minValue to maxValue inclusively
        /// </summary>
        public class RandomGenerator : IGenerator<int>
        {
            private readonly Random _rnd;
            private readonly int _minValue;
            private readonly int _maxValue;

            public RandomGenerator(int minValue, int maxValue)
            {
                _rnd = new Random();
                this._minValue = minValue;
                this._maxValue = maxValue;
            }

            public static IGenerator<int> operator +(RandomGenerator gen1, IGenerator<int> gen2)
            {
                return new AdditionGenerator(gen1, gen2);
            }

            public IEnumerator<int> GetEnumerator()
            {
                while (true)
                {
                    yield return _rnd.Next(_minValue, _maxValue + 1);
                }
            }

            IEnumerator IEnumerable.GetEnumerator()
            {
                return ((IEnumerable<int>)this).GetEnumerator();
            }

            public int Min()
            {
                return _minValue;
            }
        }
        
        /// <summary>
        /// Combines two generators through addition
        /// </summary>
        internal class AdditionGenerator : IGenerator<int>
        {
            private IGenerator<int> _gen1;
            private IGenerator<int> _gen2;

            public AdditionGenerator(Program.RandomGenerator gen1, Program.IGenerator<int> gen2)
            {
                this._gen1 = gen1;
                this._gen2 = gen2;
            }

            public IEnumerator<int> GetEnumerator()
            {
                var en1 = _gen1.GetEnumerator();
                var en2 = _gen2.GetEnumerator();
                while (true)
                {
                    var hasValue = en1.MoveNext();
                    if (hasValue != en2.MoveNext())
                    {
                        throw new InvalidOperationException("One generator stopped providing values before the other");
                    }
                    if (!hasValue)
                    {
                        yield break;
                    }
                    yield return en1.Current + en2.Current;
                }

            }

            IEnumerator IEnumerable.GetEnumerator()
            {
                return ((IEnumerable<int>)this).GetEnumerator();
            }

            public int Min()
            {
                return _gen1.Min() + _gen2.Min();
            }
        }
    }

Conclusion (so far)

I am going to think about this some more. It has a lot of potential as type abstraction, but to be honest, I deal very little in numerical values and math and statistics, so I don't see what I personally could do with this. I suspect, though, that other people might find it very useful or at least interesting. And yes, I am aware of mathematical concepts like interval arithmetic and I am sure there are a ton of existing libraries that already do something like that and much more, but I am looking at this more from the standpoint of computer science and quasi-primitive types than from a mathematical or numerical perspective. If you have any suggestions or ideas or requests, let me know!

 Intro

  When I was a kid, computers didn't have multithreading, multitasking or even multiple processes. You executed a program and it was the only program that was running. Therefore the way to do, let's say, user key input was to check again and again if there is a key in a buffer. To give you a clearer view on how bonkers that was, if you try something similar in Javascript the page dies. Why? Because the processing power to look for a value in an array is minuscule and you will basically have a loop that executes hundreds of thousand or even millions of times a second. The CPU will try to accommodate that and run at full power. You will have a do nothing loop that will take the entire capacity of the CPU for the current process. The browser would have problems handling legitimate page events, like you trying to close it! Ridiculous!

Bad solution

Here is what this would look like:

class QBasic {

    constructor() {
        this._keyBuffer=[];
        // add a global handler on key press and place events in a buffer
        window.addEventListener('keypress', function (e) {
            this._keyBuffer.push(e);
        }.bind(this));
    }

    INKEY() {
        // remove the first key in the buffer and return it
        const ev = this._keyBuffer.shift();
        // return either the key or an empty string
        if (ev) {
            return ev.key;
        } else {
            return '';
        }
    }
}

// this code will kill your CPU and freeze your page
const qb = new QBasic();
while (qb.INKEY()=='') {
 // do absolutely nothing
}

How then, should we port the original QBasic code into Javascript?

WHILE INKEY$ = ""

    ' DO ABSOLUTELY NOTHING

WEND

Best solution (not accepted)

Of course, the best solution is to redesign the code and rewrite everything. After all, this is thirty year old code. But let's imagine that, in the best practices of porting something, you want to find the first principles of translating QBasic into Javascript, then automate it. Or that, even if you do it manually, you want to preserve the code as much as possible before you start refactoring it. I do want to write a post about the steps of refactoring legacy code (and as you can see, sometimes I actually mean legacy, as in "bestowed upon by our forefathers"), but I wanted to write something tangible first. Enough theory!

Interpretative solution (not accepted, yet)

Another solution is to reinterpret the function into a waiting function, one that does nothing until a key is pressed. That would be easier to solve, but again, I want to translate the code as faithfully as possible, so this is a no-no. However, I will discuss how to implement this at the end of this post.

Working solution (slightly less bad solution)

Final solution: do the same thing, but add a delay, so that the loop doesn't use the entire pool of CPU instructions. Something akin to Thread.Sleep in C#, maybe. But, oops! in Javascript there is no function that would freeze execution for a period of time.

The only thing related to delays in Javascript is setTimeout, a function that indeed waits for the specified interval of time, but then executes the function that was passed as a parameter. It does not pause execution. Whatever you write after setTimeout will execute immediately. Enter async/await, new in Javascript ES8 (or EcmaScript 2017), and we can use the delay function as we did when exploring QBasic PLAY:

function delay(duration) {
    return new Promise(resolve => setTimeout(resolve, duration));
}

Now we can wait inside the code with await delay(milliseconds);. However, this means turning the functions that use it into async functions. As far as I am concerned, the pollution of the entire function tree with async keywords is really annoying, but it's the future, folks!

Isn't this amazing? In order to port to Javascript code that was written in 1990, you need features that were added to the language only in 2017! If you wanted to do this in Javascript ES5 you couldn't do it! The concept of software development has changed so much that it would have been impossible to port even the simplest piece of code from something like QBasic to Javascript.

Anyway, now the code looks like this:

function delay(duration) {
    return new Promise(resolve => setTimeout(resolve, duration));
}

class QBasic {

    constructor() {
        this._keyBuffer=[];
        // add a handler on every key press and place events in a buffer
        window.addEventListener('keypress', function (e) {
            this._keyBuffer.push(e);
        }.bind(this));
    }

    async INKEY() {
        // remove the first key in the buffer and return it
        const ev = this._keyBuffer.shift();
        // return either the key or an empty string
        if (ev) {
            return ev.key;
        } else {
            await delay(100);
            return '';
        }
    }
}

const qb = new QBasic();
while (qb.INKEY()=='') {
 // do absolutely nothing
}

Now, this will work by delaying for 100 milliseconds when there is nothing in the buffer. It's clearly not ideal. If one wanted to fix a problem with a loop running too fast, then the delay function should have at least been added to the loop, not the INKEY function. Using it like this you will get some inexplicable delays in code that would want to use fast key inputs. It's, however, the only way we can implement an INKEY function that will behave as close to the original as possible, which is hiring a 90 year old guy to go to a letter box and check if there is any character in the mail and then come back and bring it to you. True story, it's the original implementation of the function!

Interpretative solution (implementation)

It would have been much simpler to implement the function in a blocking manner. In other words, when called, INKEY would wait for a key to be pressed, then exit and return that key when the user presses it. We again would have to use Promises:

class QBasic {

    constructor() {
        this._keyHandler = null;
        // instead of using a buffer for keys, keep a reference
        // to a resolve function and execute it if it exists
        window.addEventListener('keypress', function (e) {
            if (this._keyHandler) {
                const handler = this._keyHandler;
                this._keyHandler = null;
                handler(e.key);
            }
        }.bind(this));
    }

    INKEY() {
        const self = this;
        return new Promise(resolve => self._keyHandler = resolve);
    }
}


const qb = new QBasic();
while ((await qb.INKEY())=='') { // or just await qb.INKEY(); instead of the loop
 // do absolutely nothing
}

Amazing again, isn't it? The loops (pun not intended) through which one has to go in order to force a procedural mindset on an event based programming language.

Disclaimer

Just to make sure, I do not recommend this style of software development; this is only related to porting old school code and is more or less designed to show you how software development has changed in time, from a period before most of you were even born.

Intro

  This post will take you on an adventure through time and sound. It will touch the following software development concepts:

  • await/async in Javascript
  • named groups in regular expressions in Javascript
  • the AudioContext API in Javascript
  • musical note theory
  • Gorillas!

  In times immemorial, computers were running something called the DOS operating system and almost the entire interface was text based. There was a way to draw things on the screen, by setting the values of pixels directly in the video memory. The sound was something generated on a "PC speaker" which was a little more than a small speaker connected to a power port and which you had to make work by handling "interrupts". And yet, since this is when I had my childhood, I remember so many weird little games and programs from that time with a lot of nostalgic glee.

  One of these games was Gorillas, where two angry gorillas would attempt to murder each other by throwing explosive bananas. The player would have to enter the angle and speed and also take into account a wind speed that was displayed as an arrow on the bottom of the screen. That's all. The sounds were ridiculous, the graphics really abstract and yet it was fun. So, as I was remembering the game, I thought: what would it take to make that game available in a modern setting? I mean, the programming languages, the way people thought about development, the hardware platform, everything has changed.

  In this post I will detail the PLAY command from the ancient programming language QBASIC. This command was being used to generate sound by instructing the computer to play musical notes on the PC speaker. Here is an example of usage:

PLAY "MBT160O1L8CDEDCDL4ECC"

  This would play the short song at the beginning of the Gorillas game. The string tells the computer to play the sound in the background, at a tempo of 160 in the first octave, with notes of an eighth of a measure: CDEDCD then end with quarter measure notes: ECC. I want to replicate this with Javascript, one because it's simpler to prototype and second because I can make the result work in this very post.

Sound and Music

  But first, let's see how musical notes are being generated in Javascript, using the audio API. First you have to create an AudioContext instance, with which you create an Oscillator. On the oscillator you set the frequency and then... after a while you stop the sound. The reason why the API seems so simplistic is because it works by creating an audio graph of nodes that connect to each other and build on each other. There are multiple ways in which to generate sound, including filling a buffer with data and playing that, but I am not going to go that way.

  Therefore, in order to PLAY in Javascript I need to translate concepts like tempo, octaves, notes and measures into values like duration and frequency. That's why we need a little bit of musical theory.

  In music, sounds are split into domains called octaves, each holding seven notes that, depending on your country, are either Do, Re, Mi, Fa, So, La, Si or A, B,C, D, E, F and G or something else. Then you have half notes, so called sharp or flat notes: A# is half a note above A and A♭ is a half note below A. A# is the same as B♭. For reasons that I don't want to even know, the octaves start with C. Also the notes themselves are not equally spaced. The octaves are not of the same size, in terms of frequency. Octave 0 starts at 16.35Hz and ends at 30.87, octave 1 ranges between 32.70 and 61.74. In fact, each octave spreads on twice as much frequency space as the one before. Each note has twice the frequency of the same note on the lower octave.

  In a more numerical way, octaves are split into 12: C, C#, D, E♭, E, F, F#, G, G#, A, B♭, B. Note (heh heh) that there are no half notes between B and C and E and F. The frequency of one of these notes is 21/12 times the one before. Therefore one can compute the frequency of a note as:

Frequency = Key note * 2n/12, where the key note is a note that you use as a base and n is the note-distance between the key note and the note you want to play.

  The default key note is A4, or note A from octave 4, at 440Hz. That means B♭ has a frequency of 440*1.059463 = 466.2.

  Having computed the frequency, we now need the duration. The input parameters for this are: tempo, note length, mode and the occasional "dot":

  • tempo is the number of quarter measures in a minute
    • this means if the tempo is 120, a measure is 60000 milliseconds divided by 120, then divided by 4, so 125 milliseconds
  • note length - the length of a note relative to a measure
    • these are usually fractions of a measure: 1, 1/2, 1/4, 1/8, 1/16, etc
  • mode - this determines a general speed of playing the melody
    • as defined by the PLAY command, you have:
      • normal: a measure is 7/8 of a default measure
      • legato: a measure is a measure
      • staccato: a measure is 3/4 of a default measure
  • dotted note - this means a specific note will be played for 3/2 of the defined duration for that note

  This gives us the formula:

Duration = note length * mode * 60000 / 4 / tempo * dotDuration

Code

  With this knowledge, we can start writing code that will interpret musical values and play a sound. Now, the code will be self explanatory, hopefully. The only thing I want to discuss outside of the audio related topic is the use of async/await in Javascript, which I will do below the code. So here it is:

class QBasicSound {

    constructor() {
        this.octave = 4;
        this.noteLength = 4;
        this.tempo = 120;
        this.mode = 7 / 8;
        this.foreground = true;
        this.type = 'square';
    }

    setType(type) {
        this.type = type;
    }

    async playSound(frequency, duration) {
        if (!this._audioContext) {
            this._audioContext = new AudioContext();
        }
        // a 0 frequency means a pause
        if (frequency == 0) {
            await delay(duration);
        } else {
            const o = this._audioContext.createOscillator();
            const g = this._audioContext.createGain();
            o.connect(g);
            g.connect(this._audioContext.destination);
            o.frequency.value = frequency;
            o.type = this.type;
            o.start();
            await delay(duration);
            // slowly decrease the volume of the note instead of just stopping so that it doesn't click in an annoying way
            g.gain.exponentialRampToValueAtTime(0.00001, this._audioContext.currentTime + 0.1);
        }
    }

    getNoteValue(octave, note) {
        const octaveNotes = 'C D EF G A B';
        const index = octaveNotes.indexOf(note.toUpperCase());
        if (index < 0) {
            throw new Error(note + ' is not a valid note');
        }
        return octave * 12 + index;
    }

    async playNote(octave, note, duration) {
        const A4 = 440;
        const noteValue = this.getNoteValue(octave, note);
        const freq = A4 * Math.pow(2, (noteValue - 48) / 12);
        await this.playSound(freq, duration);
    }

    async play(commandString) {
        const reg = /(?<octave>O\d+)|(?<octaveUp>>)|(?<octaveDown><)|(?<note>[A-G][#+-]?\d*\.?)|(?<noteN>N\d+\.?)|(?<length>L\d+)|(?<legato>ML)|(?<normal>MN)|(?<staccato>MS)|(?<pause>P\d+\.?)|(?<tempo>T\d+)|(?<foreground>MF)|(?<background>MB)/gi;
        let match = reg.exec(commandString);
        let promise = Promise.resolve();
        while (match) {
            let noteValue = null;
            let longerNote = false;
            let temporaryLength = 0;
            if (match.groups.octave) {
                this.octave = parseInt(match[0].substr(1));
            }
            if (match.groups.octaveUp) {
                this.octave++;
            }
            if (match.groups.octaveDown) {
                this.octave--;
            }
            if (match.groups.note) {
                const noteMatch = /(?<note>[A-G])(?<suffix>[#+-]?)(?<shorthand>\d*)(?<longerNote>\.?)/i.exec(match[0]);
                if (noteMatch.groups.longerNote) {
                    longerNote = true;
                }
                if (noteMatch.groups.shorthand) {
                    temporaryLength = parseInt(noteMatch.groups.shorthand);
                }
                noteValue = this.getNoteValue(this.octave, noteMatch.groups.note);
                switch (noteMatch.groups.suffix) {
                    case '#':
                    case '+':
                        noteValue++;
                        break;
                    case '-':
                        noteValue--;
                        break;
                }
            }
            if (match.groups.noteN) {
                const noteNMatch = /N(?<noteValue>\d+)(?<longerNote>\.?)/i.exec(match[0]);
                if (noteNMatch.groups.longerNote) {
                    longerNote = true;
                }
                noteValue = parseInt(noteNMatch.groups.noteValue);
            }
            if (match.groups.length) {
                this.noteLength = parseInt(match[0].substr(1));
            }
            if (match.groups.legato) {
                this.mode = 1;
            }
            if (match.groups.normal) {
                this.mode = 7 / 8;
            }
            if (match.groups.staccato) {
                this.mode = 3 / 4;
            }
            if (match.groups.pause) {
                const pauseMatch = /P(?<length>\d+)(?<longerNote>\.?)/i.exec(match[0]);
                if (pauseMatch.groups.longerNote) {
                    longerNote = true;
                }
                noteValue = 0;
                temporaryLength = parseInt(pauseMatch.groups.length);
            }
            if (match.groups.tempo) {
                this.tempo = parseInt(match[0].substr(1));
            }
            if (match.groups.foreground) {
                this.foreground = true;
            }
            if (match.groups.background) {
                this.foreground = false;
            }

            if (noteValue !== null) {
                const noteDuration = this.mode * (60000 * 4 / this.tempo) * (longerNote ? 1 : 3 / 2);
                const duration = temporaryLength
                    ? noteDuration / temporaryLength
                    : noteDuration / this.noteLength;
                const A4 = 440;
                const freq = noteValue == 0
                    ? 0
                    : A4 * Math.pow(2, (noteValue - 48) / 12);
                const playPromise = () => this.playSound(freq, duration);
                promise = promise.then(playPromise)
            }
            match = reg.exec(commandString);
        }
        if (this.foreground) {
            await promise;
        } else {
            promise;
        }
    }
}

function delay(duration) {
    return new Promise(resolve => setTimeout(resolve, duration));
}

One uses the code like this:

var player = new QBasicSound();
await player.play('T160O1L8CDEDCDL4ECC');

Note that you cannot start playing the sound directly, you need to wait for a user interaction first. An annoying rule to suppress annoying websites which would start playing the sound on load. And here is the result (press multiple times on Play for different melodies):

Javascript in modern times

There are two concepts that were used in this code that I want to discuss: named regular expression groups and async/await. Coincidentally, both are C# concepts that have crept up in the modern Javascript specifications when .NET developers from Microsoft started contributing to the language.

Named groups are something that appeared in ES2018 and it is something I've been using with joy in .NET and hated when I didn't have it in some other language. Look at the difference between the original design and the current one:

// original design
var match = /(a)bc/.exec('abcd');
if (match && match[1]) { /*do something with match[1]*/ }

// new feature
const match = /(?<theA>a)bc/.exec('abcd');
if (match && match.groups.theA) { /*do something with match.groups.theA*/ }

There are multiple advantages to this:

  • readability for people revisiting the code
  • robustness in the face of changes to the regular expression
    • the index might change if new groups are added to it
  • the code aligns with the C# code (I like that :) )

My advice is to always use named groups when using regular expressions.

Another concept is await/async. In .NET it is used to hide complex asynchronous interactions in the code and with the help of the compiler helps with all the tasks that are running at the same time. Unfortunately, in C#, that means polluting code with async keywords on all levels as async methods can only be used inside other async methods. No such qualms in Javascript.

While in .NET the await/async system runs over Task<T> methods, in Javascript it runs over Promises. Both are abstractions over work that is being done asynchronously.

A most basic example is this:

// original design
getSomethingAsync(url,function(data) {
  getSomethingElseAsync(data.url,function(data2) {
    // do something with data2
  }, errorHandler2);
},errorHandler1);

// Promises
getSomethingAsync(url)
  .then(function(data) {
    getSomethingElseAsync(data.url);
  })
  .then(function(data2) {
    // so something with data2
  })
  .catch(errorHandler);

// async/await
try {
  var data = await getSomethingAsync(url);
  var data2 = await getSomethingElseAsync(data.url);
  // do something with data2
} catch(ex) {
  errorHandler(ex);
}

You see that the await/async way looks like synchronous code, you can even catch errors. await can be used on any function that returns a Promise instance and the result of it is a non-blocking wait until the Promise resolves and returns the value that was passed to the resolve function.

If you go back to the QBasicSound class, at the end, depending on if the sound is in the foreground or background, the function is either awaiting a promise or ... just letting it run. You might also notice that I've added a delay function at the end of the code which is using setTimeout to resolve a Promise. Here is what is actually going on:

// using await
console.log(1);
await delay(1000).then(()=>console.log(2));
console.log(3);
// this logs 1,2,3


// NOT using await
console.log(1);
delay(1000).then(()=>console.log(2));
console.log(3);
// this logs 1,3,2

In the first case, the Promise that was constructed by a one second delay and then logging 2 is awaited, meaning the code waits for the result. After it is executed, 3 gets logged. In the second case, the logging of 2 is executed after one second delay, but the code does not wait for the result, therefore 3 is logged immediately and 2 comes after.

What sorcery is this?! Isn't Javascript supposed to be single threaded? How does it work? Well, consider that in the delay function, the resolve function will only be called after a timeout of one second. When executed, it starts the timeout, then reaches the end of the function. It has not been resolved yet, so it passes control back to the engine, which uses it to execute other things. When the timeout is fired, the engine takes back control, executes the resolve function, then passes control back. All of this is invisible to the user, who gets the illusion of multithreaded behavior.

Already some standard out of the box APIs are async, like fetch. In order to get an object from a REST API that is called via HTTP the code would look like this:

// fetch API
let response = await fetch('/article/promise-chaining/user.json');
let user = await response.json();

Conclusion

I spent an entire day learning about sounds and writing code that would emulate QBASIC code from a billion years ago. Who knows, maybe my next project will be to port the entire Gorillas game in Javascript. Now one can lovingly recreate the sounds of one's childhood.

Other references:

Gorillas.BAS

QBasic/Appendix

Generate Sounds Programmatically With Javascript

Musical Notes

Gorrilas game online

  So you want to use a queue, a structure that has items added at one side and removed on another, in Javascript code. Items are added to the tail of the queue, while they are removed at the head. We, Romanians, are experts because in the Communist times resources were scarce and people often formed long queues to get to them, sometimes only on the basis of rumour. They would see a line of people and ask "Don't they have meat here?" and the answer would come "No, they don't have milk here. It's next building they don't have meat at". Anyway...

  There is an option that can be used directly out of the box: the humble array. It has methods like .push (add an item), .pop (remove the latest added item - when you use it as a stack) and .shift (remove the oldest added item - when you use it as a queue). For small cases, that is all you need.

  However, I needed it in a high performance algorithm and if you think about it, removing the first element of an array usually means shifting (hence the name of the function) all elements one slot and decreasing the length of the array. Consider a million items array. This is not an option.

  One of the data structure concepts we are taught at school is the linked list. Remember that? Each item has a reference to the next (and maybe the previous) item in the list. You explore it by going from one item to the next, without indexing, and you can remove any part of the list or add to any part of the list just by changing the value of these references. This also means that for each value you want stored you have the value, the reference(s) and the overhead of handling a more complex data object. Again, consider a million numbers array. It's not the right fit for this problem.

  Only one option remains: still using an array, but moving the start and the end of the array in an abstract manner only, so that all queue/dequeue operations take no effort. This means keeping a reference to the tail and the head of the queue in relation to the length of the queue and of the underlying array.

  But first let's establish a baseline. Let's write a test and implement a queue using the default array pop/shift implementation:

// the test
const size = 100000;
const q=new Queue();
time(()=> { for (let i=0; i<size; i++) q.enqueue(i); },'Enqueue '+size+' items');
time(()=> { for (let i=0; i<size; i++) q.dequeue(i); },'Dequeue '+size+' items');
time(()=> { for (let i=0; i<size/10; i++) {
	for (let j=0; j<10; j++) q.enqueue(i);
	for (let j=0; j<9; j++) q.dequeue(i);
} },'Dequeue and enqueue '+size+' items');

// the Queue implementation
class Queue {
  constructor() {
    this._arr = [];
  }

  enqueue(item) {
    this._arr.push(item);
  }

  dequeue() {
    return this._arr.shift();
  }
}

// the results
Enqueue 100000 items, 10ms
Dequeue 100000 items, 1170ms
Dequeue and enqueue 100000 items, 19ms

  The Enqueue operation is just adding to an array, enqueuing and dequeuing by leaving an item at ever series of dequeues is slightly slower, as the amount of array shifting is negligible. Dequeuing, though, is pretty heavy. Note that increasing just a little bit the amount of items leads to an exponential increase in time:

Enqueue 200000 items, 12ms
Dequeue 200000 items, 4549ms
Dequeue and enqueue 200000 items, 197ms

  Now let's improve the implementation of the queue. We will keep enqueue using Array.push, but use a _head index to determine which items to dequeue. This means faster speed, but the queue will never shorten. It's the equivalent of Romanians getting their product, but remaining in the queue.

// the Queue implementation
class Queue {
  constructor() {
    this._arr = [];
    this._head = 0;
  }

  enqueue(item) {
    this._arr.push(item);
  }

  dequeue() {
    if (this._head>=this._arr.length) return;
    const result = this._arr[this._head];
    this._head++;
    return result;
  }
}

// the results
Enqueue 200000 items, 11ms
Dequeue 200000 items, 4ms
Dequeue and enqueue 200000 items, 11ms

  The performance has reached the expected level. Dequeuing is now even faster than enqueuing because it doesn't need to expand the array as items are added. However, for all scenarios the queue is only growing, even when dequeuing all the items. What I can do is reuse the slots of the dequeued items for the items to be added. Now it gets interesting!

  My point is that right now we can improve the functionality of our queue by replacing dequeued but still stored items with newly enqueued items. That is the equivalent of Romanians leaving the queue only after they get the meat and a new Romanian comes to take their place. If there are more people coming than getting served, then people that got their meat will all leave and we can just add people to the tail of the queue.

  So let's recap the algorithm:

  • we will use an array as a buffer
  • the queue items start at the head and end at the tail, but wrap around the array buffer
  • whenever we add an item, it will be added in the empty space inside the array and the tail will increment
  • if there is no empty space (queue length is the same as the array length) then the array will be rearranged so that it has space for new itms
  • when we dequeue, the item at the head will be returned and the head incremented
  • whenever the head or tail reach the end of the array, they will wrap around

Some more improvements:

  • if we enqueue a lot of items then dequeue them, the array will not decrease until we dequeue them all. An improvement is to rearrange the array whenever the queue length drops below half of that of the array. It will add computation, but reduce space.
  • when we make space for new items (when the array size is the same as the one of the logical queue) we should add more space than just 1, so I will add the concept of a growth factor and the smallest size increase.

Here is the code:

/**
 * A performant queue implementation in Javascript
 *
 * @class Queue
 */
class Queue {

    /**
     *Creates an instance of Queue.
     * @memberof Queue
     */
    constructor() {
        this._array = [];
        this._head = 0;
        this._tail = 0;
        this._size = 0;
        this._growthFactor = 0.1;
        this._smallestSizeIncrease = 64;
    }

    /**
     * Adding an iterator so we can use the queue in a for...of loop or a destructuring statement [...queue]
     */
    *[Symbol.iterator]() {
        for (let i = 0; i < this._size; i++) {
            yield this.getAt(i);
        }
    }

    /**
     * Returns the length of the queue
     *
     * @readonly
     * @memberof Queue
     */
    get length() {
        return this._size;
    }

    /**
     * Get item based on item in the queue
     *
     * @param {*} index
     * @returns
     * @memberof Queue
     */
    getAt(index) {
        if (index >= this._size) return;
        return this._array[(this._head + index) % this._array.length];
    }

    /**
     * Gets the item that would be dequeued, without actually dequeuing it
     *
     * @returns
     * @memberof Queue
     */
    peek() {
        return this.getAt(0);
    }

    /**
     * Clears the items and shrinks the underlying array
     */
    clear() {
        this._array.length = 0;
        this._head = 0;
        this._tail = 0;
        this._size = 0;
    }

    /**
     * Adds an item to the queue
     *
     * @param {*} obj
     * @memberof Queue
     */
    enqueue(obj) {
        // special case when the size of the queue is the same as the underlying array
        if (this._size === this._array.length) {
            // this is the size increase for the underlying array
            const sizeIncrease = Math.max(this._smallestSizeIncrease, ~~(this._size * this._growthFactor));
            // if the tail is behind the head, it means we need to move the data from the head to 
            // the end of the array after we increase the array size
            if (this._tail <= this._head) {
                const toMove = this._array.length - this._head;
                this._array.length += sizeIncrease;
                for (let i = 0; i < toMove; i++) {
                    this._array[this._array.length - 1 - i] = this._array[this._array.length - 1 - i - sizeIncrease];
                }
                this._head = (this._head + sizeIncrease) % this._array.length;
            }
            else
            // the array size can just increase (head is 0 and tail is the end of the array)
            {
                this._array.length += sizeIncrease;
            }
        }
        this._array[this._tail] = obj;
        this._tail = (this._tail + 1) % this._array.length;
        this._size++;
    }

    /**
     * Removed the oldest items from the queue and returns it
     *
     * @returns
     * @memberof Queue
     */
    dequeue() {
        if (this._size === 0) {
            return undefined;
        }
        const removed = this._array[this._head];
        this._head = (this._head + 1) % this._array.length;
        this._size--;
        // special case when the size of the queue is too small compared to the size of the array
        if (this._size > 1000 && this._size < this._array.length / 2 - this._smallestSizeIncrease) {
            if (this._head<this._tail) {
                this._array = this._array.slice(this._head,this._tail);
            } else {
                this._array=this._array.slice(this._head, this._array.length).concat(this._array.slice(0,this._tail));
            }
            this._head = 0;
            this._tail = 0;
        }   
        return removed;
    }
}

  Final notes:

  • there is no specification on how an array should be implemented in Javascript, therefore I've used the growth factor concept, just like in C#. However, according to James Lawson, the array implementation is pretty smart in modern Javascript engines, we might not even need it.
  • the optimization in dequeue might help with space, but it could be ignored if what you want is speed and don't care about the space usage
  • final benchmarking results are:
    Enqueue 200000 items, 15ms, final array size 213106
    Dequeue 200000 items, 19ms, final array size 1536
    Dequeue and enqueue 200000 items, 13ms, final array size 20071

  While working on my pet project Linqer (LINQ for Javascript and Typescript) I've spent quite a lot of time improving the performance of the Quicksort algorithm I am using for .orderBy. Therefore I am publishing it here, even if you could extract it just the same from the Linqer sources, with limited discussion on what is going on.

  Why

  First, why use it at all? Doesn't Javascript have the .sort method in the Array class? What's wrong with that?

  The answer is that the implementation for sort is different from browser to browser, or better said, from Javascript engine to Javascript engine. In Chrome, for example, the algorithm used is insertion sort, which is simple, in place, stable and reasonably fast. It is optimized for the most common usage: small arrays that need to be sorted for UI purposes and such. However, when using large arrays, the algorithm doesn't perform as well as one might expect.

  For Linqer I had an additional reason, because I would use ordering followed by skip and take methods that limited the scope of the need for sorting. Imagine a million items array that I wanted ordered and then needed the first ten items. Sorting the entire thing for just ten items would have been overkill. The default .sort function doesn't have parameters for such scenarios.

  And there is another reason: the default function used to compare array items is alphanumeric. [1, 2, 10] would get ordered as [1, 10, 2].

  Second, why Quicksort? There are a bunch of sorting algorithms out there. Mergesort, Heapsort, Radixsort, Timsort, Selectionsort. What's so special about Quicksort.

  I have to admit that I went for it by googling fast sorting algorithm. It does have "quick" in the name, doesn't it? I also found it elegant and easy to comprehend. And for my particular scenario, I liked that it used a divide et impera strategy which allowed me to ignore parts of the array if I didn't need the items there. In other words, it's very well suited both as a general sorting algorithm and a partial sorting algorithm.

  What

  I would like to tell you that it's simple to explain what Quicksort does, but it requires some amount of attention and time. In general terms, it chooses an arbitrary item (called a pivot) then orders the remaining items relative to the pivot, in two so called partitions: the smaller items to the left, the larger to the right. Then it repeats the process for each of the two sides. How the pivot is chosen and how the partitions are handled is what differentiates Quicksort algorithms and determines their performance.

  It is an in place algorithm, meaning it doesn't copy the array in some other type of structure and instead it moves items around inside it. It is not a stable algorithm, meaning the order of "equal" items is not preserved. The average computational complexity is O(n log n), with the worst cases O(n^2). The space complexity is harder to determine. Most people say it is O(1) because it uses no additional data structures, but that is not really correct. Being a recursive algorithm, the call stack gets used quite a lot, an invisible storage that should be computed in the data complexity.

  Unfortunately, the worst case scenarios are also very common: already sorted arrays and arrays filled with the same value. There are various optimizations to be used in order to handle this sort of thing. Also, Quicksort is efficient with large quantities of data, but less so with small numbers of items.

  How

  Finally, we get to the code. The _quicksort function receives:

  • an array
  • left and right index values determining the inclusive area that will be sorted (usually 0 and array.length-1)
  • a comparer function (item1,item2)=> 1, 0 or -1 and that defaults to _defaultComparer which tries to sort items based on the > and < operators
  • min and max index values determining the window of the array that we need to have sorted

The left and right indexes determine which section (before the sort) of the array will be sorted, the min and max indexes determine which items I am interested in (after the sort). This allows me to skip ordering partitions that are outside my area of interest.

As I said, the pivot choice is important. Some strategies are very popular:

  • the last item in the array as the pivot
    • this is the strategy used in the original incarnation of Quicksort
    • leads to very poor performance when the array is already sorted
  • the median item
    • this suggests parsing the array in order to get the value, implying extra computation
    • it only makes sense when the values in the array are numbers
  • the average between the first, the last and the middle item
    • it only makes sense when the values in the array are numbers
  • the item that is in the middle of the array
    • this is the one that I am using
  • a random item in the array
    • this makes the algorithm escape scenarios where the performance would be bad
    • the outcome of the sorting is unpredictable in terms of time used and stability of items
  • multiple pivots
    • an interesting concept, but one that complicated the algorithm too much for comfort

Then there is the matter of the partitioning. I've used an optimization that involves two indexes, one at the start the other at the end of a partition, coming toward each other and swapping items that are on the wrong side of the pivot. In some implementations, if the pivot is the last item, the partitioning is from one side only. In others, multiple indexes are used to handle multiple pivots.

In most implementations, the algorithm recurses on _quicksort, but I refactored it to only recurse on the partitioning. Then, because I didn't want to get stack overflows when bad data was used, I've eliminated the recursion and instead used a stack of my own where the partitions to be sorted are stored and wait their turn. This is where the data complexity comes around. In my case I was using a little more data than I actually needed, because I was adding partitions to the stack and also incrementing the index of the current partition, meaning the stack array grew with handled partitions. Even if there is no computation performance benefit, I've optimized this as well by adding a queueIndex which is used to recycle the slots in the partition array that are behind the partitionIndex. New partitions are being added behind the partitionIndex and queueIndex is incremented. When the loop reaches the last partition in the stack, a new loop is started with the partitions from 0 to queueIndex. Thus, for a ten million item array the partition stack rarely goes above 40000 in length.

A further optimization is to use insertion sort on partitions that have become too small (under 64 items). It irks me to have had to do this, I would have liked to use a "pure" algorithm, but this improved the performance and minimized the size of the partition stack.

The Code

That's about it. Here is the code:

    function _insertionsort(arr, leftIndex, rightIndex, comparer) {
        for (let j = leftIndex; j <= rightIndex; j++) {
            const key = arr[j];
            let i = j - 1;
            while (i >= leftIndex && comparer(arr[i], key) > 0) {
                arr[i + 1] = arr[i];
                i--;
            }
            arr[i + 1] = key;
        }
    }
    function _swapArrayItems(array, leftIndex, rightIndex) {
        const temp = array[leftIndex];
        array[leftIndex] = array[rightIndex];
        array[rightIndex] = temp;
    }
    function _partition(items, left, right, comparer) {
        const pivot = items[(right + left) >> 1];
        while (left <= right) {
            while (comparer(items[left], pivot) < 0) {
                left++;
            }
            while (comparer(items[right], pivot) > 0) {
                right--;
            }
            if (left < right) {
                _swapArrayItems(items, left, right);
                left++;
                right--;
            }
            else {
                if (left === right)
                    return left + 1;
            }
        }
        return left;
    }
    const _insertionSortThreshold = 64;
    function _quicksort(items, 
                        left, right, comparer = _defaultComparer,
                        minIndex = 0, maxIndex = Number.MAX_SAFE_INTEGER) {
        if (!items.length)
            return items;
        const partitions = [];
        partitions.push({ left, right });
        while (partitions.length) {
            ({ left, right } = partitions.pop());
            if (right - left < _insertionSortThreshold) {
                _insertionsort(items, left, right, comparer);
                continue;
            }
            const index = _partition(items, left, right, comparer);
            if (left < index - 1 && index - 1 >= minIndex) {
                partitions.push({ left, right: index - 1 });
            }
            if (index < right && index < maxIndex) {
                partitions.push({ left: index, right });
            }
        }

        return items;
    }
    _defaultComparer = (item1, item2) => {
        if (item1 > item2)
            return 1;
        if (item1 < item2)
            return -1;
        return 0;
    };

So I was trying to optimize a sorting algorithm, only the metrics did not make any sense. On the test page I had amazing performance, on the other it was slow as hell. What could possibly be the problem?

The difference between the two tests was that one was sorting inline (normal array reading and writing) and the other was using a more complex function and iterated a data source. So I went to test the performance of iterating itself.

The code tests the speed of adding all the items of a large array in three cases:

  • a classic for...in loop that increments an index and reads the array at that index
  • a for...of loop that iterates the items of the array directly
  • a for...of loop that iterates over a generator function that yields the values of the array
time(()=>{ let sum=0; for (let i=0; i<arr.length; i++) sum+=arr[i]; },'for in');
time(()=>{ let sum=0; for (const v of arr) sum+=v; },'iterator for of');
time(()=>{ let sum=0; for (const v of (function*(){ for (let i=0; i<arr.length; i++) yield arr[i]; })()) sum+=v; },'generator for of');

time is a function I used to compute the speed of the execution. The array is 100 million integers. And here are the results:

for in: 155.12999997008592
for of: 1105.7250000303611
for of: 2309.88499999512

I haven't yet processed what it means, but I really thought using an iterator was going to be at least as fast as a for loop that uses index accessing to read values. Instead there is a whooping 7 to 14 times decrease in speed.

So from now on I will avoid for...of in high performance scenarios.

Just a thing I learned today: using the bitwise not operator (~) on a number in Javascript ignores its fractional part (it converts it to integer first), therefore using it twice gives you the integer part of original number. Thanks to fetishlace for clarifications.

Notes:

  • this is equivalent to (int)number in languages that support the int type
  • this is equivalent to Math.trunc for numbers in the integer range
  • this is equivalent to Math.floor only for positive numbers in the integer range

Examples:
~~1.3 = 1
~~-6.5432 = -6
~~(2 ** 32 + 0.5) = 0
~~10000000000 = 1410065408

P.S. If you don't know what ** is, it's called the Exponentiation operator and it came around in Javascript ES2016 (ES7) and it is the equivalent of Math.pow. 2 ** 3 = Math.pow(2,3) = 8

  I was looking into the concept of partial sorting, something that would help in a scenario where you want the k smaller or bigger items from an array of n items and k is significantly smaller than n. Since I am tinkering with LInQer, my implementation for LINQ methods in Javascript, I wanted to tackle the OrderBy(...).Take(k) situation. Anyway, doing that I found out some interesting things.

  First, the default Javascript array .sort function has different implementations depending on browser and by that I mean different sort algorithms. Chrome uses insertion sort and Firefox uses merge sort. None of them is Quicksort, the one that would function best when the number of items is large, but that has worst case complexity O(n^2) when the array is already sorted (or filled with the same value).

I've implemented a custom function to do Quicksort and after about 30000 items it becomes faster than the default one.  For a million items the default sort was almost three times slower than the Quicksort implementation. To be fair, I only tested this on Chrome. I have suspicions that the merge sort implementation might be better.

  I was reporting in a previous version of this post that QuickSort, implemented in Javascript, was faster than the default .sort function, but it seems this was only an artifact of the unit tests I was using. Afterwards, I've found an optimized version of quicksort and it performed three times faster on ten million integers. I therefore conclude that the default sort implementation leaves a lot to be desired.

  Second, the .sort function is by default alphanumeric. Try it: [1,2,10].sort() will return [1,10,2]. In order to do it numerically you need to hack away at it with [1,2,10].sort((i1,i2)=>i1-i2). In order to sort the array based on the type of item, you need to do something like: [1,2,10].sort((i1,i2)=>i1>i2?1:(i1<i2?-1:0)).

  Javascript has two useful functions defined on the prototype of Object: toString and valueOf. They can also be overwritten, resulting in interesting hacks. For some reason, the default .sort function is calling toString on the objects in the array before sorting and not valueOf. Using the custom comparers functions above we force using valueOf. I can't test this on primitive types though (number, string, boolean) because for them valueOf cannot be overridden.

  And coming back to the partial sorting, you can't do that with the default implementation, but you can with Quicksort. Simply do not sort any partition that is above and below the indexes you need to get the items from. The increases in time are amazing!

  There is a difference between the browser implemented sort algorithms and QuickSort: they are stable. QuickSort does not guarantee the order of items with equal sort keys. Here is an example: [1,3,2,4,5,0].sort(i=>i%2) results in [2,4,0,1,3,5] (first even numbers, then odd, but in the same order as the original array). The QuickSort order depends on the choice of the pivot in the partition function, but assuming a clean down the middle index, the result is [4,2,0,5,1,3]. Note that in both cases the requirement has been met and the even numbers come first.

Update: The npm package can now be used in Node.js and Typescript (with Intellisense). Huge optimizations on the sorting and new extra functions published.

Update: Due to popular demand, I've published the package on npm at https://www.npmjs.com/package/@siderite/linqer. Also, while at it, I've moved the entire code to Typescript, fixed a few issues and also create the Linqer.Slim.js file, that only holds the most common methods. The minimized version can be found at https://siderite.github.io/LInQer/LInQer.slim.min.js and again you can use it freely on your websites. Now, I hope you can use it in Node JS as well, I have no idea how to test it and frankly I don't want to. I've also added a tests.slim.html that only tests the slim version of Linqer. This version is now official and I will try to limit any breaking changes from now on. WARNING: Now the type is Linqer.Enumerable, not just Enumerable as before. If you are already using it in code, just do const Enumerable = Linqer.Enumerable; and you are set to go.

Update: LInQer can now be found published at https://siderite.github.io/LInQer/LInQer.min.js and https://siderite.github.io/LInQer/LInQer.extra.min.js and you can freely use it in your web sites. Source code on GitHub.

Update: All the methods in Enumerable are now implemented, and some extra ones as well. Optimizations have been developed for operations like orderBy().take() and .count().

  This entire thing started from a blog post that explained something that seems obvious in retrospect, but you have to actually think about it to realise it: Array functions in Javascript may seem similar to LInQ methods in C#, but they work with arrays! Every time you filter or you map something you create a new array! The post suggested creating functions that would use the modern Javascript features of iterators and generator functions and supplant the standard Array functions. It was brilliant!

  And then the post abruptly veered and swerved into a tree. The author suggested we add the pipe operator to Javascript, just like they have in functional programming languages, so that we can use those static functions with hash characters as placeholders and... ugh! It was horrid! So I decided to solve the problem in a way that was more compatible with my own sensibilities: make it work just like LInQ (or at least like the Java streams).

This is how LInQer was born. You can find the sources on GitHub at /Siderite/LInQer. The library introduces a class called Enumerable, which can wrap any iterable (meaning arrays, but also generator functions or anything that supports for...of), then expose the methods that the Enumerable class in .NET exposes. There is even a unit test file called tests.html. Open it in the browser and see the supported methods. As you can see in the image of the blog post, the same logic (filtering, mapping and slicing a 10 million item array) took 2000ms using standard Array functions and 150ms using the Enumerable class.

"But wait! It says there that Enumerable exposes static methods! What exactly did you improve?", you will ask. In .NET we have something called "extension methods", which are indeed static methods that the language understands apply to specific classes or interfaces and allows you to call them like you would instance methods. So a method that looks like public static int Sum(this IEnumerable<int> enumerable) can be used statically Enumerable.Sum(arr) or like an instance method arr.Sum(). It's syntactic sugar. But enough about C#, in Javascript I've implemented Enumerable as a wrapper, as Javascript doesn't know about interfaces or static extension methods. This class then exposes prototype functions that work the same way as in LInQ.

Here is where the magic happens:

Enumerable.prototype = {
  [Symbol.iterator]() {
    const iterator = this._src[Symbol.iterator].bind(this._src);
    return iterator();
  },

In other words I wrap _src (the original iterable object) in my Enumerable by exposing the same iterator for both! Now both initial object and its wrapper can be used in for...of loops. The difference is that the wrapper now exposes all that juicy functionality. Let's see a simple example: select (which is the logical equivalent of Array.map):

select(op) {
  _ensureFunction(op);
  const gen = function* () {
    for (const item of this) {
      yield op(item);
    }
  }.bind(this);
  return new Enumerable(gen());
},

Here I am constructing a generator function which I bind to the Enumerable instance so that inside it 'this' is always that instance. Then I am returning the wrapper over the generator. In the function I am yielding the result of the op function on each item. Because of how generator functions work, only while iterating will it need to do its work, meaning that if I use the Enumerable into a for...of loop and breaking after three items, the op function will only be applied on those items, not on all of them. In order to use the Enumerable object in regular code, that works with arrays, you just use .toArray and you are done!

Let's see the code in the tests used to compare the standard Array function use with Enumerable:

// this is the large array we are using in both code blocks:
  const largeArray = Array(10000000).fill(10);

// Standard Array functions
  const someCalculation = largeArray
                             .filter(x=>x===10)
                             .map(x=>'v'+x)
                             .slice(100,110);
// Enumerable
  const someCalculation = Linqer.Enumerable.from(largeArray)
                             .where(x=>x===10)
                             .select(x=>'v'+x)
                             .skip(100)
                             .take(10)
                             .toArray();

There are differences from the C# Enumerable, of course. Some things don't make sense, like thenBy after orderBy. It can be done, but it's complicated and in my career I've only used thenBy twice! toDictionary, toHashSet, toLookup, toList have no meaning in Javascript. Use instead toMap, toSet, toObject, toArray. Last, but not least... join. Join is so complicated to use in LInQ that I almost never used it. Also, joining is usually done in the database, so I rarely needed it. I didn't see a point in implementing it. Cast and AsEnumerable also didn't make sense. But I implemented them anyway! :) Cast and OfType are using either a class or a string to determine if an item is "of type" and join works just like in C#.

But don't fret. Any function that you want to add to this can simply be added by your own code into Enumerable.prototype! So if you really need something custom, it's easy to add without modifying the original library.

There is one disadvantage for the prototype approach and the reason why the initial article suggested to use standalone functions: tree-shaking, the fancy word used to express the automatic elimination of unused code. But I think there is a solution for that, one that I won't implement since I believe the library is small enough and minified and compressed it will be very small indeed. The solution would involve separating each of the LINQ methods (or categories of methods, like first, firstOrDefault, last and lastOrDefault, for example) in different files. Then you can use only what you need and the files would attach the functions to the Enumerable prototype. 

In fact, there are a lot of LINQ methods I have never had use for, stuff like intersect and except or append and prepend. It makes sense to create a core Enumerable class and then add other stuff only when required. But as I said, it's small enough to not matter. As such I separated the functionality in Typescript files that contain the basic functionality, the complete functionality and some extra functions. The Javascript result is a Linqer, a Linqer.slim, a Linqer.extra and a Linqer.all, covering the entire spectrum of needs. It would have been user unfriendly to put every little thing in its own file.

Bottom line, I hope you find this library useful or at least it inspires you as it did me when I've read the original blog post from Dan Shappir.

P.S. I am not the only one that had this idea, you might also want to look at linq.js which uses a completely different approach, using regular expressions. I think the closest to what LINQ should be is Manipula, that uses custom iterators for each operation rather than use the same class. Another possibility is linq-collections, which does the work via TypeScript, apparently. Also linq.es6... I am sure there are more examples if one looks closely. Feel free to let me know if you did or know of similar work and also please give me whatever feedback you see fit. I would like this library to be useful, not just a code example for my blog.

I hope you at least heard of the concept of Unit Testing as it is one of the principal pillars of software development. Its purpose is to automatically check the functionality of isolated components of your code, but it adds a lot of benefits:

  • your code becomes more modular - you only worry about the code you change
  • your code becomes more readable - clear dependency chain and tests demonstrate in code what was the purpose of particular features
  • you gain confidence that your code works as you are making changes to it - refactoring without unit tests in place is usually dangerous
  • changing any component with another implementation does not affect the overall application

This post is a companion to the Programming a simple game in pure HTML and Javascript in the sense that it is that game that we will be testing. Also, as per the requirements of that project, we will not be using any of the numerous unit testing frameworks available for Javascript code.

When we left off the development of the game we had three files:

  • complementary.html - the structure of the page
  • complementary.css - the visual design of the page components
  • complementary.js - all the code of the game, including the initialization and the game starting bit

In order to test our individual components, we need to separate them. So let's split complementary.js in four files:

  • complementary.js - just the game start (instantiating Game and initializing it)
  • game.js - the Game class
  • color.js - the Color class
  • elements.js - the custom HTML elements and their registration

Obviously the HTML will change to load all of these Javascript files. When we get to modules, this will become a non issue.

There are things that we could test on the custom HTML elements, but let's leave that aside. Also the two lines in complementary.js will not need testing. The simplest component should be the first to be tested and that is Color.

We start by creating a new HTML file (color tests.html) and we fill it with code that checks the Color class works as expected. First the code and then the discussion:

<html>

<head>
    <script src="color.js"></script>
    <script>
        // this can easily be changed to display a nice report in this page
        const assert ={
            true:(value, message)=> {
                if (value) {
                    console.log('Test PASSED ('+message+')');
                } else {
                    console.warn('Test FAILED');
                    alert(message);
                    throw new Error(message);
                }
            }
        };
    </script>
</head>

<body>
    <script>
        // Arrange
        const color1 = new Color(123);
        const color2 = new Color(123);
        const color3 = new Color(234);
        // Act
        const equalsWorks = color1.equals(color2);
        const notEqualsWorks = !color1.equals(color3);
        // Assert
        assert.true(equalsWorks,'Expected two colors initialized with the same value to be equal');
        assert.true(notEqualsWorks,'Expected two colors initialized with different values not to be equal');
    </script>

    <script>
        // Arrange
        const color = new Color();
        const doubleInvertedColor = color.complement().complement();
        // Act
        const complementAndEqualsWork = color.equals(doubleInvertedColor);
        // Assert
        assert.true(complementAndEqualsWork,'Expected the complementary or a complementary color to be the original color');
    </script>

    <script>
        // Arrange
        const acolor = new Color(0x6789AB);
        const stringColor = acolor.toString();
        // Act
        const toStringWorks = stringColor==='#6789ab';
        // Assert
        assert.true(toStringWorks,'Expected the HTML representation of the color to be #6789ab and it was '+stringColor);
    </script>

</body>

</html>

A test should follow the AAA structure:

  • Arrange - sets up the necessary items for the test to run
    • instantiate classes to be tested
    • mock functionality of dependencies - we will see this when we test Game
  • Act - executes the code intended to be tested and acquires results
  • Assert - verifies that the test results are the ones expected

Normally, a framework would take tests written in a certain way and then produce some sort of report, with nice green and red rows, with messages, with information on where errors occurred and so on. However, I intend to demonstrate the basics of unit testing, so no framework is actually needed.

A unit test:

  • is a piece of code (who unit tests the unit test?!)
  • it requires effort, it is just as prone to bugs as normal code and requires maintenance just like any other code (shit doesn't just happen, it takes time and effort)
  • it requires infrastructure that uses it to periodically test your changes (either someone does it manually or there is some setup to run it automatically and display/email the report)

In the code above I created a script tag for each test in which I am following AAA to create Color instances and then check their functionality does what it should. A very basic assert object is handling the reporting part. It remains homework for the reader to update that part or to plug in an existing unit testing framework.

The Color class is very simple:

  • it supports initializing with a color integer representing the RGB values of the color
  • toString method that returns the HTML representation of the color
  • complement method returns the complement of the color
  • equals method checks if two colors are equal

There are no external dependencies, meaning that it needs nothing from the outside in order to work. Game, for example, requires Color, which is a dependency for Game.

Open the color tests.html file in the browser and open the development tools (F12 or Ctrl-Shift-I) and refresh the page. You should see in the console that all the tests passed. Change something in a test so it fails and it should both throw an error in the console and show you a dialog with the failing test.

Now, let's test Game. The code of the game tests.html file:

<html>

<head>
    <script src="game.js"></script>
    <script>
        // this can easily be changed to display a nice report in this page
        const assert ={
            true:(value, message)=> {
                if (value) {
                    console.log('Test PASSED ('+message+')');
                } else {
                    console.warn('Test FAILED');
                    alert(message);
                    throw new Error(message);
                }
            }
        };
    </script>
</head>

<body>
    <script>
        // Arrange
        const game = new Game();
        // Act
        // Assert
    </script>

</body>

</html>

I used the same structure, the same assert object, only I loaded game.js instead of color.js. Then I wrote a test in which I do nothing than instantiate a Game. If you execute this page it will work just fine. No errors because we have not, in fact, executed anything. We need to execute .init(document), remember?

And now it becomes apparent why I chose to initialize the document from init instead of using window.document in my code. window.document is now the document of the test page, it has no complementary-board element in it. We haven't even defined any custom HTML elements or a Color class. In fact, we can now test that calling init with no parameter will fail:

    <script>
        // Arrange
        const game = new Game();
        // Act
        let anyError = null;
        try {
            game.init();
        } catch(error) {
            anyError = error;
        }
        // Assert
        assert.true(anyError!=null,'Expected the init method of a Game class to fail if run with no parameters ('+anyError+')');
    </script>

And, indeed, if we open the page now we get a console log like this: 

Test PASSED (Expected the init method of a Game class to fail if run with no parameters (TypeError: Cannot read property 'addEventListener' of undefined))

You just learned another important characteristic of a unit test: it tests both what should work and what shouldn't. This is one of the reasons why unit testing is hard. For each piece of code you need to test when it works and when it fails as expected. Now we have to test how Game should work, and that means we get into mocking.

Mocking is when you replace a dependency with something that looks exactly the same, but does something else. For unit tests mock objects need to do the simplest things and those things must be predictable.

Let's see how one of these tests would look:

    <script>
        // Arrange
        const mockDoc = {
            addEventListener:function() {
            }
        };
        const game2 = new Game();
        // Act
        let anyError2 = null;
        try {
            game2.init(mockDoc);
        } catch(error) {
            anyError2 = error;
        }
        // Assert
        assert.true(anyError2==null,'Expected the init method of a Game class to not fail if run with correct parameters ('+anyError2+')');
    </script>

Just by providing an object with an addEventListener function, the initialization of the game now works. mockDoc is a mocked document and this is called mocking.

Let's look at how would a "happy path" test look, one that assumes everything goes correctly and moves through and entire flow:

    <script>
        // Arrange
        const mockDoc3 = {
            testData : {},
            addEventListener:function(eventName,eventHandler) {
                this.testData.eventName = eventName;
                this.onLoad = eventHandler;
            },
            getElementsByTagName:function(tagName) {
                this.testData.tagName = tagName;
                return [this.mockBoard];
            },
            mockBoard: {
                setChoiceHandler:function(handler) {
                    this.choiceHandler=handler;
                }
            }
        };
        const game3 = new Game();
        // Act
        let anyError3 = null;
        try {
            game3.init(mockDoc3);
        } catch(error) {
            anyError3 = error;
        }
        Math = {
            random:function() {
                return 0.4;    
            },
            floor:function(x) { return x; },
            round:function(x) { return x; },
            pow:function(x,p) { if (p==1) return x; else throw new Error('Expected 1 as the exponent, not '+p); }
        };
        Color = {
            index:0,
            new:function(value) {
                return {
                    val: value,
                    equals: function(x) { return x.val==this.val; },
                    complement: function() { return Color.new(1000-this.val); }
                };
            },
            random:function() {
                this.index++;
                return Color.new(this.index*10);
            }
        }
        mockDoc3.onLoad();
        mockDoc3.mockBoard.choiceHandler(3);
        // Assert
        assert.true(anyError3==null,'Expected the init method of a Game class to not fail if run with correct parameters ('+anyError3+')');
        assert.true(mockDoc3.testData.eventName==='DOMContentLoaded','DOMContentLoaded was not handled!');
        assert.true(mockDoc3.testData.tagName==='complementary-board','Game is not looking for a complementary-board element');
        assert.true(game3._roundData.guideColor.val === 10,'Guide color object expected to have val=10 ('+JSON.stringify(game3._roundData.guideColor)+')');
        assert.true(game3._roundData.tries.size === 1,'Expected 1 unsuccessful try ('+JSON.stringify(game3._roundData.tries)+')');
        assert.true(game3._roundData.tries.has(3),'Expected unsuccessful try with value of 3 ('+JSON.stringify(game3._roundData.tries)+')');
        assert.true(game3._log.length === 0,'Expected no score after one unsuccessful try ('+JSON.stringify(game3._log)+')');
        // Act 2 (heh!)
        mockDoc3.mockBoard.choiceHandler(2);
        // Assert 2
        assert.true(game3._roundData.guideColor.val === 60,'Guide color object expected to have val=60 ('+JSON.stringify(game3._roundData.guideColor)+')');
        assert.true(game3._roundData.tries.size === 0,'Expected no unsuccessful tries after correct response ('+JSON.stringify(game3._roundData.tries)+')');
        assert.true(game3._log.length === 1,'Expected one item of score after correct answer ('+JSON.stringify(game3._log)+')');
        assert.true(game3._log[0] === 50,'Expected 50 as the score after one fail and one correct answer ('+JSON.stringify(game3._log)+')');
        
    </script>

There is a lot to unpack here, including the lazy parts of the test. Here are some of the issues with it:

  • it doesn't follow the AAA pattern, it asserts, then acts again, then asserts again
    • while this works, it doesn't encapsulate testing of a single feature
    • it is the equivalent of the classes or methods with multiple responsibilities from normal code
    • the correct way to do it is to write another test, have two choiceHandler calls in the Act part and assert only that particular path
  • it tests internal functionality
    • in order to write the test I had to look at how the Game class works internally and then tailor the test so it works
    • it accesses private data like _log and _roundData
  • the Game class did not abstract all of its dependencies
    • this is painfully obvious when mocking the Math object - in Javascript this is easy, but in other languages the Math functionality comes as an interface (a declaration of available members with no implementation)
    • this is not a test problem, but a Game problem which makes testing tedious
  • test contains logic
    • look at the Math and Color mocks, how they compute values and return objects
  • some values there are particularly chosen to "work"
    • have you noticed how random returns 0.4 so that multiplied with 5 (number of choices) it returns 2, which then is equal with the correct choice?
  • test is not independent
    • Math and Color are replaced with some static objects, thus changing the environment for following tests

But it works, even if it's not very well written. Here are some of its good features:

  • it tests a full game round with one bad and one good choice
  • it doesn't try to go around randomness by adding more code in the assertion part
    • it could have easily gone that way in order to determine the correct color in a random list, thus replicating or reverse engineering the Game logic into the test
  • all the expected results are completely predictable (the score values, the indexes, etc)
  • it tests only Game functionality, not others
    • I could have not mocked Color, loading color.js and thus relying in a single test on functionality from a dependency

The lessons learned from this tell us that both the Game code and the test code could have been written better in regard to maintainability, with dependencies clearly declared, easy to mock or replace. In fact, the greatest gain of a company with hiring a senior developer is not on how well code works, but how much time is saved and how much risk is avoided when the code is written with separation of concerns and unit testing in mind.

It is funny, but when a piece of code is not testable, writing a single unit test forces you to refactor it into a good shape. The rest of the tests only test functionality, as the class has been rewritten with testing in mind. So that is my advice: whenever you write something, even a silly game like Complementary, write one "happy path" unit test per class.

Homework for you: rewrite the Game class and its test class so that testing becomes easier and more correct.

The code for this series of posts can be found at https://github.com/Siderite/Complementary

  I was helping a friend with basic programming and I realized that I've been so caught up with the newest fads and development techniques that I've forgotten about simple programming, for fun, with just the base principles and tools provided "out of the box". This post will demonstrate me messing up writing a game using HTML and Javascript only.

Mise en place

This French phrase is used in professional cooking to represent the preparation of ingredients and utensils before starting the actual cooking. We will need this before starting developing our game:

  • description: the game will show a color and the player must choose from a selection of other colors the one that is complementary
    • two colors are complementary if when they are mixed, they cancel each other out, resulting in a grayscale "color" like white, black or some shade of gray. Wait! Was that the metaphor in Fifty Shades of Grey?
  • technological stack: HTML, Javascript, CSS
    • flavor of Javascript: ECMAScript 2015 (also known as ES6)
    • using modules: no - this would be nice, but modules obey CORS, so you won't be able to run it with the browser from the local file system.
    • unit testing: yes, but we have to do it as simply as possible (no external libraries)
  • development IDE: Visual Studio Code
    • it's free and if you don't like it, you can just use Notepad to the same result
  • source control: Git (on GitHub)

Installing Visual Studio Code

Installing VS Code is just as simple as downloading the installer and running it.

Then, select the Open Folder option, create a project folder (let's call it Complementary), then click on Select Folder.

The vanilla installation will help you with syntax highlighting, code completion, code formatting.

Project structure

For starters we will need the following files:

  • complementary.html - the actual page that will be open by the browser
  • complementary.js - the Javascript code
  • complementary.css - the CSS stylesheet

Other files will be added afterwards, but this is the most basic separation of concerns: code and data in the .js file, structure in .html and presentation in .css.

Starting to code

First, let's link the three files together by writing the simplest HTML structure:

<html>
    <head>
        <link rel="stylesheet" href="complementary.css"/>
        <script src="complementary.js"></script>
    </head>
    <body>
        
    </body>
</html>

This instructs the browser to load the CSS and JS files. 

In the Javascript file we encapsulate out logic into a Game class:

"use strict";
class Game {
  init(doc) {
    this._document = doc;
    this._document.addEventListener('DOMContentLoaded',this.onLoad.bind(this),false);
  }
  onLoad() {

  }
}

const game=new Game();
game.init(document);

We declared a class (a new concept in Javascript ES6) and a method called init that receives a doc. The idea here is that when the script is loaded, a new Game will be created and the initialization function will receive the current document so it can interact with the user interface. We used the DOMContentLoaded event to call onLoad only when the page document object model (DOM) has been completely loaded, otherwise the script would run before the elements have been loaded.

Also, not the use of the bind method on a function. addEventListener expects a function as the event handler. If we only specify this.onLoad, it will run the function, but with the this context of the event, which would be window, not our game object. this.onLoad.bind(this), on the other hand, is a function that will be executed in the context of our game.

Now, let's consider how we want to game to play out:

  • a guide color must be shown
    • this means the color needs to be generated
  • a list of colors to choose from must be displayed
    • colors need to be generated
    • one color needs to be complementary to the guide color
    • color elements need to respond to mouse clicks
  • a result must be computed from the chosen color
    • the outcome of the user choice must be displayed
    • the score will need to be calculated

This gives us the structure of the game user interface. Let's add:

  • a guide element
  • a choice list element
  • a score element
<html>
    <head>
        <link rel="stylesheet" href="complementary.css"/>
        <script type="module" src="complementary.js"></script>
    </head>
    <body>
        <div id="guideColor"></div>
        <div id="choiceColors"></div>
        <div id="score"></div>
    </body>
</html>

Note that we don't need to choose how they look (that's the CSS) or what they do (that's the JS).

This is a top-down approach, starting from user expectations and then filling in more and more details until it all works out.

Let's write the logic of the game. I won't discuss that too much, because it's pretty obvious and this post is about structure and development, not the game itself.

"use strict";
class Game {
    constructor() {
        // how many color choices to have
        this._numberOfChoices = 5;
        // the list of user scores
        this._log = [];
    }
    init(doc) {
        this._document = doc;
        this._document.addEventListener('DOMContentLoaded', this.onLoad.bind(this), false);
    }
    onLoad() {
        this._guide = this._document.getElementById('guideColor');
        this._choices = this._document.getElementById('choiceColors');
        // one click event on the parent, but event.target contains the exact element that was clicked
        this._choices.addEventListener('click', this.onChoiceClick.bind(this), false);
        this._score = this._document.getElementById('score');
        this.startRound();
    }
    startRound() {
        // all game logic works with numeric data
        const guideColor = this.randomColor();
        this._roundData = {
            guideColor: guideColor,
            choiceColors: this.generateChoices(guideColor),
            tries: new Set()
        };
        // only this method transforms the data into visuals
        this.refreshUI();
    }
    randomColor() {
        return Math.round(Math.random() * 0xFFFFFF);
    }
    generateChoices(guideColor) {
        const complementaryColor = 0xFFFFFF - guideColor;
        const index = Math.floor(Math.random() * this._numberOfChoices);
        const choices = [];
        for (let i = 0; i < this._numberOfChoices; i++) {
            choices.push(i == index
                ? complementaryColor
                : this.randomColor());
        }
        return choices;
    }
    refreshUI() {
        this._guide.style.backgroundColor = '#' + this._roundData.guideColor.toString(16).padStart(6, '0');
        while (this._choices.firstChild) {
            this._choices.removeChild(this._choices.firstChild);
        }
        for (let i = 0; i < this._roundData.choiceColors.length; i++) {
            const color = this._roundData.choiceColors[i];
            const elem = this._document.createElement('span');
            elem.style.backgroundColor = '#' + color.toString(16).padStart(6, '0');
            elem.setAttribute('data-index', i);
            this._choices.appendChild(elem);
        }
        while (this._score.firstChild) {
            this._score.removeChild(this._score.firstChild);
        }
        const threshold = 50;
        for (let i = this._log.length - 1; i >= 0; i--) {
            const value = this._log[i];
            const elem = this._document.createElement('span');

            elem.className = value >= threshold
                ? 'good'
                : 'bad';
            elem.innerText = value;
            this._score.appendChild(elem);
        }
    }
    onChoiceClick(ev) {
        const elem = ev.target;
        const index = elem.getAttribute('data-index');
        // just a regular expression test that the attribute value is actually a number
        if (!/^\d+$/.test(index)) {
            return;
        }
        const result = this.score(+index);
        elem.setAttribute('data-result', result);
    }
    score(index) {
        const expectedColor = 0xFFFFFF - this._roundData.guideColor;
        const isCorrect = this._roundData.choiceColors[index] == expectedColor;
        if (!isCorrect) {
            this._roundData.tries.add(index);
        }
        if (isCorrect || this._roundData.tries.size >= this._numberOfChoices - 1) {
            const score = 1 / Math.pow(2, this._roundData.tries.size);
            this._log.push(Math.round(100 * score));
            this.startRound();
        }
        return isCorrect;
    }
}

const game = new Game();
game.init(document);

This works, but it has several problems, including having too many responsibilities (display, logic, handling clicks, generating color strings from numbers, etc).

And while we have the logic and the structure, the display leaves a lot to be desired. Let's fix this first (I am terrible with design, so I will just dump the result here and it will be a homework for the reader to improve on the visuals).

First, I will add a new div to contain the three others. I could work directly with body, but it would be ugly:

<html>

<head>
    <link rel="stylesheet" href="complementary.css" />
    <script src="complementary.js"></script>
</head>

<body>
    <div class="board">
        <div id="guideColor"></div>
        <div id="choiceColors"></div>
        <div id="score"></div>
    </div>
</body>

</html>

Then, let's fill in the CSS:

body {
    width: 100vw;
    height: 100vh;
    margin: 0;
}
.board {
    width:100%;
    height:100%;
    display: grid;
    grid-template-columns: 50% 50%;
    grid-template-rows: min-content auto;
}
#score {
    grid-column-start: 1;
    grid-column-end: 3;
    grid-row: 1;
    display: flex;
    flex-direction: row;
    flex-wrap: nowrap;
}
#score span {
    display: inline-block;
    padding: 1rem;
    border-radius: 0.5rem;
    background-color: darkgray;
    margin-left: 2px;
}
#score span.good {
    background-color: darkgreen;
}
#score span.bad {
    background-color: red;
}
#guideColor {
    grid-column: 1;
    grid-row: 2;
}
#choiceColors {
    grid-column: 2;
    grid-row: 2;
    display: flex;
    flex-direction: column;
}
#choiceColors span {
    flex-grow: 1;
    cursor: pointer;
}
#choiceColors span[data-result=false] {
    opacity: 0.3;
}

I used a lot of flex and grid to display things.

The game should now do the following:

  • displays a left side color
  • displays five rows of different colors in the right side
  • clicking on any of them modifies the score (each wrong choice halves the maximum score of 100)
  • when there are no more moves left or the correct choice is clicked, the score is added to a list at the top of the board
  • the score tiles are either green (score>=50) or red

However, I am dissatisfied with the Javascript code. If Game has too many responsibilities it is a sign that new classes need to be created.

Refactoring the code

First, I will encapsulate all color logic into a Color class.

class Color {
    constructor(value = 0  /* black */) {
        this._value = value;
    }
    toString() {
        return '#' + this._value.toString(16).padStart(6, '0');
    }
    complement() {
        return new Color(0xFFFFFF - this._value);
    }
    equals(anotherColor) {
        return this._value === anotherColor._value;
    }
    static random() {
        return new Color(Math.round(Math.random() * 0xFFFFFF));
    }
}

This simplifies the Game class like this:

class Game {
    constructor() {
        // how many color choices to have
        this._numberOfChoices = 5;
        // the list of user scores
        this._log = [];
    }
    init(doc) {
        this._document = doc;
        this._document.addEventListener('DOMContentLoaded', this.onLoad.bind(this), false);
    }
    onLoad() {
        this._guide = this._document.getElementById('guideColor');
        this._choices = this._document.getElementById('choiceColors');
        // one click event on the parent, but event.target contains the exact element that was clicked
        this._choices.addEventListener('click', this.onChoiceClick.bind(this), false);
        this._score = this._document.getElementById('score');
        this.startRound();
    }
    startRound() {
        // all game logic works with numeric data
        const guideColor = Color.random();
        this._roundData = {
            guideColor: guideColor,
            choiceColors: this.generateChoices(guideColor),
            tries: new Set()
        };
        // only this method transforms the data into visuals
        this.refreshUI();
    }
    generateChoices(guideColor) {
        const complementaryColor = guideColor.complement();
        const index = Math.floor(Math.random() * this._numberOfChoices);
        const choices = [];
        for (let i = 0; i < this._numberOfChoices; i++) {
            choices.push(i == index
                ? complementaryColor
                : Color.random());
        }
        return choices;
    }
    refreshUI() {
        this._guide.style.backgroundColor = this._roundData.guideColor.toString();
        while (this._choices.firstChild) {
            this._choices.removeChild(this._choices.firstChild);
        }
        for (let i = 0; i < this._roundData.choiceColors.length; i++) {
            const color = this._roundData.choiceColors[i];
            const elem = this._document.createElement('span');
            elem.style.backgroundColor = color.toString();
            elem.setAttribute('data-index', i);
            this._choices.appendChild(elem);
        }
        while (this._score.firstChild) {
            this._score.removeChild(this._score.firstChild);
        }
        const threshold = 50;
        for (let i = this._log.length - 1; i >= 0; i--) {
            const value = this._log[i];
            const elem = this._document.createElement('span');

            elem.className = value >= threshold
                ? 'good'
                : 'bad';
            elem.innerText = value;
            this._score.appendChild(elem);
        }
    }
    onChoiceClick(ev) {
        const elem = ev.target;
        const index = elem.getAttribute('data-index');
        // just a regular expression test that the attribute value is actually a number
        if (!/^\d+$/.test(index)) {
            return;
        }
        const result = this.score(+index);
        elem.setAttribute('data-result', result);
    }
    score(index) {
        const expectedColor = this._roundData.guideColor.complement();
        const isCorrect = this._roundData.choiceColors[index].equals(expectedColor);
        if (!isCorrect) {
            this._roundData.tries.add(index);
        }
        if (isCorrect || this._roundData.tries.size >= this._numberOfChoices - 1) {
            const score = 1 / Math.pow(2, this._roundData.tries.size);
            this._log.push(Math.round(100 * score));
            this.startRound();
        }
        return isCorrect;
    }
}

But it's still not enough. Game is still doing a lot of UI stuff. Can we fix that? Yes, with custom HTML elements!

Here is the code. It looks verbose, but what it does is completely encapsulate UI logic into UI elements:

class GuideColor extends HTMLElement {
    set color(value) {
        this.style.backgroundColor = value.toString();
    }
}

class ChoiceColors extends HTMLElement {
    connectedCallback() {
        this._clickHandler = this.onChoiceClick.bind(this);
        this.addEventListener('click', this._clickHandler, false);
    }
    disconnectedCallback() {
        this.removeEventListener('click', this._clickHandler, false);
    }
    onChoiceClick(ev) {
        const elem = ev.target;
        if (!(elem instanceof ChoiceColor)) {
            return;
        }
        const result = this._choiceHandler(elem.choiceIndex);
        elem.choiceResult = result;
    }
    setChoiceHandler(handler) {
        this._choiceHandler = handler;
    }
    set colors(value) {
        while (this.firstChild) {
            this.removeChild(this.firstChild);
        }
        for (let i = 0; i < value.length; i++) {
            const color = value[i];
            const elem = new ChoiceColor(color, i);
            this.appendChild(elem);
        }
    }
}

class ChoiceColor extends HTMLElement {
    constructor(color, index) {
        super();
        this.color = color;
        this.choiceIndex = index;
    }
    get choiceIndex() {
        return +this.getAttribute('data-index');
    }
    set choiceIndex(value) {
        this.setAttribute('data-index', value);
    }
    set choiceResult(value) {
        this.setAttribute('data-result', value);
    }
    set color(value) {
        this.style.backgroundColor = value.toString();
    }
}

class Scores extends HTMLElement {
    set scores(log) {
        while (this.firstChild) {
            this.removeChild(this.firstChild);
        }
        for (let i = log.length - 1; i >= 0; i--) {
            const value = log[i];
            const elem = new Score(value);
            this.appendChild(elem);
        }
    }
}

class Score extends HTMLElement {
    constructor(value) {
        super();
        this.innerText = value;
        this.className = value > 50
            ? 'good'
            : 'bad';
    }
}

class Board extends HTMLElement {
    constructor() {
        super();
        this._guide = new GuideColor();
        this._choices = new ChoiceColors();
        this._score = new Scores();
    }
    connectedCallback() {
        this.appendChild(this._guide);
        this.appendChild(this._choices);
        this.appendChild(this._score);
    }
    setChoiceHandler(handler) {
        this._choices.setChoiceHandler(handler);
    }
    set guideColor(value) {
        this._guide.color = value;
    }
    set choiceColors(value) {
        this._choices.colors = value;
    }
    set scores(value) {
        this._score.scores = value;
    }
}

window.customElements.define('complementary-board', Board);
window.customElements.define('complementary-guide-color', GuideColor);
window.customElements.define('complementary-choice-colors', ChoiceColors);
window.customElements.define('complementary-choice-color', ChoiceColor);
window.customElements.define('complementary-scores', Scores);
window.customElements.define('complementary-score', Score);

With this, the HTML becomes:

<html>

<head>
    <link rel="stylesheet" href="complementary.css" />
    <script src="complementary.js"></script>
</head>

<body>
    <complementary-board>
    </complementary-board>
</html>

and the CSS:

body {
    width: 100vw;
    height: 100vh;
    margin: 0;
}
complementary-board {
    width:100%;
    height:100%;
    display: grid;
    grid-template-columns: 50% 50%;
    grid-template-rows: min-content auto;
}
complementary-scores {
    grid-column-start: 1;
    grid-column-end: 3;
    grid-row: 1;
    display: flex;
    flex-direction: row;
    flex-wrap: nowrap;
}
complementary-score {
    display: inline-block;
    padding: 1rem;
    border-radius: 0.5rem;
    background-color: darkgray;
    margin-left: 2px;
}
complementary-score.good {
    background-color: darkgreen;
}
complementary-score.bad {
    background-color: red;
}
complementary-guide-color {
    grid-column: 1;
    grid-row: 2;
}
complementary-choice-colors {
    grid-column: 2;
    grid-row: 2;
    display: flex;
    flex-direction: column;
}
complementary-choice-color {
    flex-grow: 1;
    cursor: pointer;
}
complementary-choice-color[data-result=false] {
    opacity: 0.3;
}

Next

In the next blog posts we will see how we can test our code (we have to make it more testable first!) and how we can use Git as source control. Finally we should have a working game that can be easily modified independently: the visual design, the working code, the structural elements.