and has 0 comments
As part of a self imposed challenge to write a blog post about code on average every day for about 100 days, I am discussing a link that I've previously shared on Facebook, but that I liked so much I feel the need to contemplate some more: The Caching Antipattern

Basically, what is says is that caching is done wrong in any of these cases:
  • Caching at startup - thus admitting that your dependencies are too slow to begin with
  • Caching too early in development - thus hiding the performance of the service you are developing
  • Integrated cache - cache is embedded and integral in the service code, thus breaking the single responsibility principle
  • Caching everything - resulting in an opaque service architecture and even recaching. Also caching things you think will be used, but might never be
  • Recaching - caches of caches and the nightmare of untangling and invalidating them in cascade
  • Unflushable cache - no method to invalidate the cache except restarting services

The alternatives are not to cache and instead know your data and service performance and tweak them accordingly. Try to take advantage of client caches such as the HTTP If-Modified-Since header and 304-not-modified return code whenever possible.

I've had the opportunity to work on a project that did almost everything in the list above. The performance bottleneck was the database, so the developers embedded an in code memory cache for resources that were not likely to change. Eventually other anti-patterns started to emerge, like having a filtered call (let's say a call asking for a user by id) getting all users and then selecting the one that had that id. Since it was in memory anyway, "getting" the entire list of records was just getting a reference to an already existing data structure. However, if I ever wanted to get rid of the cache or move it in an external service, I would have had to manually look for all cases of such presumptions in code and change them. If I wanted to optimize a query in the database I had no idea how to measure the performance as actually used in the product, in fact it led to a lack of interest in the way the database was used and a strong coupling of data provider implementation to the actual service code. Why use memory tables for often queried data? We have it cached anyway.

My take on this is that caching should always be separate from the service code. Have the code work, as slow as the data provider and external services allow it, then measure the performance and add caching where actually needed, as a separate layer of indirection. These days there are so many ways to do caching, from in memory tables in SQL Server, to distributed memory caches provided as a service by most cloud providers - such as Memcached or Redis in AWS, to content caches like Akamai, to html output caches like Varnish and to client caches, controlled by response and request headers like in the suggestion from the original article. Adding your own version is simply wasteful and error prone. Just like the data provider should be used through a thin interface that allows you to replace it at will, the caching layer should also be plug and play, thus allowing your application to remain unchanged, but able to upgrade any of its core features when better alternatives arrive.

There is also something to be said about reaching the limit of your resources. Let's say you cache everything your clients ask for in memory, when they ask for it. At one time or another you might reach the upper limit of your memory. At this time the cache should not fail, instead it should clear the data least used or the oldest inserted or something like that. A cache is not something that is supposed to hold all your data, only the part of it that is most efficient, performance wise, and it should never ever bring more problems like memory overflow crashes. Eek!

Now, I need to find a suitable name for my caching layer invalidation manager ;)

Other interesting resources about caching:
Cache (computing)
Caching Best Practices
Cache me if you can Powerpoint presentation
Caching guidance
Caching Techniques

The first thing that strikes anyone starting to use another IDE than the one they are used to is that all the key bindings are wrong. So they immediately google something like "X key bindings for Y" and they usually get an answer, since developers switching IDEs and preferring one in particular is quite a common situation. Not so with Eclipse. You have to install software, remove settings then still modify stuff. I am going to give you the complete answer here on how to switch Eclipse key bindings to the ones you are used to in Visual Studio.

Step 1
First follow the instructions in this Stack Overflow answer: How to Install Visual Studio Key Bindings in Eclipse (Helios onwards)
Short version: Go to Help → Install New Software, select your version in the Work with box, wait until the list populates, check the box next to Programming Languages → C/C++ Development Tools and install (with restart). After that go to Window → Preferences → General → Keys and change the Scheme in a dropdown to Microsoft Visual Studio.

Step 2
When Eclipse starts it shows you a Welcome screen. Disable the welcome screen by checking the box from the bottom-right corner and restart Eclipse. This is to avoid Ctrl-arrows not working in the editor as explained in this StackOverflow answer.

Step 3
While some stuff does work, others do not. It is time to go to Window → Preferences → General → Keys and start changing key bindings. It is a daunting task at first, since you have to find the command, set the shortcut in the zillion contexts that are available and so on. The strategy I found works best is this:
  • Right click on whatever item you want to affect with the keyboard shortcut
  • Find in the context menu whatever command you want to do
  • Remember the keyboard shortcut
  • Go to the key preferences and replace that shortcut everywhere (the text filter in the key bindings dialog allows searching for keyboard shortcuts)

You might want to share or at least backup your keyboard settings. No, the Export CSV option in the key bindings dialog gives you a file you can't import. The solution, as detailed here is to go to File → Export or Import → General → Preferences and work with .epf files. And if you think it gives you a nice list of key bindings that you can edit with a file editor, think again. The format holds the key binding scheme name, then only the custom changes, in a file that is what .ini and .xml would have if they decided on having children.

Now, the real decent thing would be to not go through Step 1 and instead just start from the default bindings and change them according to Visual Studio (2016, not 2005!!) and then export the .epf file in order for all people to enjoy a simple and efficient method of reaching their goal. I leave this as an exercise for the reader.

A short list of shortcuts that I found missing from the Visual Studio schema: rename variable on F2, go to declaration on F12, Ctrl-Shift-F for search across files, Ctrl-Minus to navigate backward ... and more to come I am sure.

and has 2 comments
Ugh! I will probably be working in Java for a while. Or I will kill myself, one of the two. So far I hate Eclipse, I can't even write code and I have to blog simple stuff like how to do regular expressions. Well, take it as a learning experience! Exiting my comfort zone! 2017! Ha, ha haaaaa [sob! sob!]

In Java you use Pattern to do regex:
Pattern p = Pattern.compile("a*b");
Matcher m = p.matcher("aaaaab");
boolean b = m.matches();

So let's do some equivalent code:

.NET:
var regDate=new Regex(@"^(?<year>(?:19|20)?\d{2})-(?<month>\d{1,2})-(?<day>\d{1,2})$",RegexOptions.IgnoreCase);
var match=regDate.Match("2017-02-14");
if (match.Success) {
var year=match.Groups["year"];
var month=match.Groups["month"];
var day=match.Groups["day"];
// do something with year, month, day
}

Java:
Pattern regDate=Pattern.compile("^(?<year>(?:19|20)?\\d{2})-(?<month>\\d{1,2})-(?<day>\\d{1,2})$", Pattern.CASE_INSENSITIVE);
Matcher matcher=regDate.matcher("2017-02-14");
if (matcher.find()) {
String year=matcher.group("year");
String month=matcher.group("month");
String day=matcher.group("day");
// do something with year, month, day
}

Notes:

The first thing to note is that there is no verbatim literal support in Java (the @"string" format from .NET) and there is no "var" (one always has to specify the type of the variable, even if it's fucking obvious). Second, the regular expression object Pattern doesn't do things directly, instead it creates a Matcher object that then does operations. The two bits of code above are not completely equivalent, as the Success property in a .NET Match object holds the success of the already performed operation, while .find() in the Java Matcher object actually performs the match.

Interestingly, it seems that Pattern is automatically compiling the regular expression, something that .NET must be directed to do. I don't know if the same term means the same thing for the two frameworks, though.

Another important thing is that it is more efficient to reuse matchers rather than recreate them. So when you want to use the matcher on another string, use matcher.reset("newstring").

And lastly, the string class itself has quick and dirty regular expression methods like .matches, replaceFirst and .replaceAll. The matches method only returns a bool if the string is a perfect match (equivalent to a Pattern match with ^ at the beginning and $ at the end).

and has 0 comments
I'll be honest, I only started reading the Culture series because Elon Musk named his rockets after ships in the books; and I started with The Player of Games, because the first book was in an inconvenient ebook format. So here I was, posed to be amazed by the wonderful and famous universe created by Iain M. Banks. And it completely bored me.

The book is written in a style reminiscent of Asimov, but even older feeling, even if it was published in 1997. My mind made the connection with We, by Zamyiatin, which was published in 1921. Characters are not really developed, they are described in few details that pertain to the subject of the story. They then act and talk, occasionally the book revealing some things that seemed smart to the author when he wrote it. Secondary characters have it even worse, with the most extensively (and uselessly) cared for attribute being a long name composed of various meaningless words. The hero of the story is named Chiark-Gevantsa Jernau Morat Gurgeh dam Hassease, for example, and the sentient drone that accompanies him is Trebel Flere-Imsaho Ephandra Lorgin Estral. Does anyone care? Nope.

But wait, Asimov wrote some brilliant books, didn't he? Maybe the style is a bit off, but the world and the idea behind the story must be great, if everybody acclaims the Culture series. Nope, again. The universe is amazingly conservative, with the important actors being either humanoid or machine, and acting as if of similar intellect. The entire premise of the book is that a human is participating in a game contest with aliens, and even engages in sexual flirting and encounters with them, which felt really uninspired and even insipid. I mean, compare this with other books released in 1997, like The Neutronium Alchemist (The Night's Dawn Trilogy, #2) by Peter F. Hamilton or 3001: The Final Odyssey (Space Odyssey, #4), by Arthur C. Clarke or even Slant, by Greg Bear. Compared to these The Player of Games feels antiquated, bland. Imagine an entire book about Data fighting Sirna Kolrami in a game of Stratagema. Boring.

Perhaps the most intriguing part of the book is also the least explored: the social and moral landscape of the Culture, a multispecies conglomerate that seems to have grown above the need for stringent laws or moral rules. Since everybody can change their chemistry and body shape and have enough resources to have no need for money, they do whatever they please when they please it, as long as it doesn't disturb others too much. Now this threshold is never explained in more words than a few paragraphs. The anarchistic nature of the post scarcity society Banks described, and indeed the rest of the book, felt like a stab of our current hierarchical and rule based order, but it was a weak stab, a near miss, a mere tickle that went ignored.

Bottom line: my expectations for well renowned books may be unreasonable high, and maybe if I would have read The Game Player when I was a kid, I would have liked it. However, I wasn't a kid in 1997 and I chose to read it now, when it feels even more obsolete and bland.

and has 0 comments

The unthinkable happened and I couldn't finish a Brandon Sanderson book. True, I had no idea The Bands of Mourning was the sixth in a series, but when I found out I thought it was a good idea to read it and see if it was worth it reading the whole Mistborn series, for which Sanderson is mostly known. Well, if the other books in the series are like this one, it's kind of boring.

I didn't feel like the book was bad, don't get me wrong, it was just... painfully average. Apparently in the Mistborn universe there are people that have abilities, like super powers, and other that have even stronger powers, but use metal as fuel. Different metals give different powers. That was intriguing, I bought the premise, I wanted to see it used in an interesting way. Instead I get a main character that is also a lord and a policeman, who is solving crime with the help of a funny sidekick at the request of the gods, who are only people who have ascended into godhood, rather than the creators of the entire universe. The crime fighting lord kind of soured the whole deal for me, but I was ready to see more and get into the mood of things. I couldn't. Apparently, the only way people have thought to fight people who can affect metal is aluminium bullets, which is terribly expensive, or complex devices that nullify their power. Apparently bows and arrows or wooden bullets are beyond their imagination.

But the worst sin of the book, other than kind of recycling old ideas and having people behave stupidly is having completely unsympathetic characters. I probably would have been invested more if I would have read the first five books of the series first, but as it is, I thought all of the main characters were artificially weird, annoying and uninteresting.

Bottom line: around halfway into the book, which is short by Sanderson standards anyway, I gave up. There are so many books in the world, I certainly don't need to read this one. The Wikipedia article for the book says: Sanderson wrote the first third of Shadows of Self between revisions of A Memory of Light. However, after returning to the book in 2014 Sanderson found it difficult to get back into writing it again. To refresh himself on the world and characters, Sanderson decided to write its sequel Bands of Mourning first and at the end of 2014 he turned both novels in to his publisher. So the author was probably distracted when he wrote this book, perhaps the others are better, but as such I find it difficult to motivate myself to try reading them.

Uitindu-ma la protestele de ieri am fost surprins ca nimeni nu face legatura dintre altercatiile cu fortele de ordine si Revolutia din 1989, singura perioada in care tin minte ca ar mai fi fost asa ceva in Romania. Ieri se vorbea de Colectiv, de cit de nesimtiti sint la PSD, ca jos Dragnea. A fost Colectiv atit de departe incit nu mai tinem minte cum era? Lumea era in strada scandind impotriva clasei politice in general. Acolo nu s-au bagat ultrasii sau jandarmii, iar lumii ii era lehamite de orice forma de politica. Acum, insa, lupta e polarizata, jos unul, sus altul, si am cazut iar in ciclul ala puturos din care nu am mai iesit din '90 citeva decenii: un permanent vot impotriva, punindu-ne bolnav sperantele in cealalta directie, ca si cum citeva rocade intre partide ar fi rezolvat ceva. La Revolutie am dat jos un sistem, iar acum, cred eu, orice mai prejos de atit este un rateu gigantic.

De aceea nu o sa ma vedeti prin piete scandind impotriva unuia sau altuia. Sint toti la fel. Singura solutie este castrarea politica: sa nu mai aiba nimeni posibilitatea de a da legi nediscutate, sa poata introduce oricine o lege sau un veto la o lege cu un anumit numar de semnaturi, sa eliminam posibilitatea, prin Constitutie, ca un presedinte sa tina parlamentul blocat sau ca un parlament sa se joace cu legislatia pe cintecul vreunui partid sau ca DNAul sa tina parti si tot asa. Sa tragem la raspundere oamenii pentru vorbele, promisiunile si faptele lor. Nu cu legi si inchisori, ci public, colectiv. Cumva am uitat ca alegerile politice sint doar o abstractie a vointei populare care se poate schimba in orice moment.

Ce faci cu cineva care ti-a inselat increderea? Nu i-o mai acorzi. Daca ii dai cheile de la casa si te fura, ii iei cheile! Poate il si bati mar, dar in mod clar nu il mai lasi in casa ta. Solutia nu e nici sa dai imediat cheile altuia, ci sa le tii tu si gata.

Repet: Protestele de dupa incendiul de la Colectiv erau o explozie de indignare impotriva intregii clase politice, Revolutia de la 1989 si ce a urmat imediat, singura perioada in care imi aduc aminte sa fi fost jandarmi cu tunuri cu apa si gaze folosite impotriva manifestantilor in Romania, a fost o explozie de indignare impotriva intregului sistem politic. Ma pis pe manifestantii din Piata Victoriei daca tot ce vor este sa il dea jos pe Dragnea cind pleaca de la servici, daca asta e toata ambitia lor.

and has 0 comments
Brandon Sanderson does not disappoint with the sequel to Way of Kings. Quite the opposite, in fact, weaving more and more into the vast tapestry that is the world of the Stormlight Archive series. The characters converge towards a point in time and space where everything and anything will be decided, the fate of the entire world, with just a few courageous people standing between life and complete desolation.

Words of Radiance focuses more on the main characters, with less distractions that might take the reader out of the flow of the story. However, even if the scope of their achievements explodes, the power of their stories loses a bit of the desperation and energy from the first book. We no longer have powerless broken people trying to survive, but magical beings full of strength doing extraordinary things. Ironically, it is their success that makes them less easy to identify and empathize with. The author throws challenges in front of them, but they seem inconsequential compared to the ones in Way of Kings. I feel like he has grown attached to them and finds it difficult to torture them as a good writer should. On the other hand Sanderson is a positive person, most of this writing being lighthearted and less dark and brooding, so this is not a disappointment.

The climax is a gigantic clash between forces that have slowly grown since the beginning of the series. Sanderson does a wonderful job tying the separate strands of his world into a single story, maybe a bit too much so. The Roshar and Helaran connections felt a bit strained, not unlike Luke Skywalker discovering his greatest ally and greatest enemy are family members. The author better be careful not to put immense effort to create a vast universe, only to shrink it by mistake by connecting everything with everything and everybody with everybody.

And again the final pages of the book feel weak, as they come after the powerful climax, yet they are necessary to tie in some story arks and seed the beginning of others. Yes, the book ends with a promise that what happened in it is just the mere beginning, a small part of the larger picture, so expect little closure. Sanderdon is a prolific author and I am sure he will write the next books in the series fast enough to keep me engaged, but be aware the series is planned to be at least ten main books with about just as much companion stories and novels. This... will take a while. Oathbringer, the third book, is scheduled to be released in November 2017.

Bottom line: I recommend the book and the series and the author. No fantasy reader should ignore Brandon Sanderson if they are anything like me. Just make sure you are ready to get invested in the story only to wait every year for the next chapter to be released.

and has 1 comment

I am hearing more and more this expression that leaves me baffled: "Check your privilege!". It is directed at us, White men, by women, colored folk and gays. It is intended to make us aware of our superior position in order for us to feel guilty over it. Really? That's what you got?

First of all, shit doesn't just happen: it takes time and effort! Do you think that God bestowed our supremacy onto us or something? No! If you believe that you have bought into all the stories we've fed you. We worked hard to get where we are! What have you done? Black people have been the majority of people on Earth for millions of years, but when does humanity grow exponentially together with the living standards? That's right! When the White Man takes control. Women have been ruling the Stone Age for millennia. Where did that get us? Nowhere, that's where! Stone fashion didn't do well, did it? And now you have the gall to ask us to check our privilege? Now the shoe is on the other foot and you are sour about it. Deal with it! Facts, bitches! Not alternative ones, either. The White Man management has brought this enterprise to new heights. Everything you have now is the direct consequence of our leadership. You are beneath us because we put you there!

Every time you people complain you call yourselves "minorities". No, you're not! Women are more numerous than men and White people are fewer than blacks and browns and yellows and whatever else there is on this planet. You know who's a minority? White Men! And still the privilege is yours, since we clearly allow you to exist and complain. The only group of people that have consistently been persecuted and have been a lot fewer than other folk are the homos. They are the only ones who have the right to whine. That's why everybody says whining is gay. It's true!

Yet when we complain, we are derided! You really think it is easy to keep under boot a majority of people on Earth. You think hitting women or slaves is fun? It fucking stings! You have to take all empathy and push it way way down, swallow your tears and do the right thing for everybody, because in the end we have led human kind into its Golden Age, all through our sacrifice. And if you don't like it, that's too bad, but it's mostly your fault, anyway. You make us behave like that, even when we hate it, because you keep getting above your station.

When the most powerful man on Earth is a White Man who rightfully knows the truth about the world and our place at its helm, you act all outraged. We even allowed you to vote the person you wanted and still you chose him. Oh, he lies, you say. He's a sexist racist White Man who twists the truth to further his needs. Have you even met politicians before? They are mostly White and mostly male because, statistically proven, we rule the world best. If you do something, at least do it right!

So you check *your* privilege! You get to complain, to fight for your rights, to live, all the while basking into the glory of the White Man and reaping the fruits of his labor and sacrifice. We carry you into tomorrow like a cross on Golgotha, never complaining, being spit at all the way up, but up we climb and high we reach. If you want to get to where we are, work for it like we do. Enslave some people, cull others, smack some around in the name of God. It's not fun, but it needs doing, for the betterment of humanity as a whole. In the end, you are where you are because you know it's the right place for you, otherwise you would have done something about it. You slack away while we run things for you, it's just the way of the world. Start complaining after a few million years, when you get your turn.

Often we need to attach functions on Javascript events, but we need them to not be executed too often. Mouse move or scroll events can fire several times a second and executing some heavy computation directly would make everything slow and unresponsive. That's why we use a method called debounce, that takes your desired function and returns another function that will only get executed so many times in a time interval.



It has reached a certain kind of symmetry, so I like it this way. Let me explain how I got to it.

First of all, there is often a similar function used for debouncing. Everyone remembers it and codes it off the top of the head, but it is partly wrong. It looks like this:
function debounce(fn, wait) {
var timeout=null;
return function() {
var context=this;
var args=arguments;
var f=function(){ fn.apply(context,args); };
clearTimeout(timeout);
timeout=setTimeout(f,wait);
};
}
It seems OK, right? Just extend the time until the function gets executed. Well, the problem is that the first time you call the function you will have to wait before you see any result. If the function is called more often than the wait period, your code will never get executed. That is why Google shows this page as *the* debounce reference: JavaScript Debounce Function. And it works, but good luck trying to understand its flow so completely that you can code it from memory. My problem was with the callNow variable, as well as the rarity of cases when I would need to not call the function immediately the first time, thus making the immediate variable redundant.

So I started writing my own code. And it looked like the "casual" debounce function, with an if block added. If the timeout is already set, then just reset it; that's the expected behavior. When isn't this the expected behavior? When calling it the first time or after a long period of inactivity. In other words when the timeout is not set. And the code looked like this:
function debounce(fn, wait) {
var timeout=null;
return function() {
var context=this;
var args=arguments;
var f=function(){ fn.apply(context,args); };
if (timeout) {
clearTimeout(timeout);
timeout=setTimeout(f,wait);
} else {
timeout=setTimeout(function() {
clearTimeout(timeout);
timeout=null;
});
f();
}
};
}

The breakthrough came with the idea to use the timeout anyway, but with an empty function, meaning that the first time it is called, the function will execute your code immediately, but also "occupy" the timeout with an empty function. Next time it is called, the timeout is set, so it will be cleared and reset with a timeout using your initial code. If the interval elapses, then the timeout simply gets cleared anyway and next time the call of the function will be immediate. If we abstract the clearing of timeout and the setting of timeout in the functions c and t, respectively, we get the code you saw at the beginning of the post. Note that many people using setTimeout/clearTimeout are in the scenario in which they set the timeout immediately after they clear it. This is not always the case. clearTimeout is a function that just stops a timer, it does not change the value of the timeout variable. That's why, in the cases when you want to just clear the timer, I recommend also setting the timeout variable to null or 0.

For the people wanting to look cool, try this version:
function debounce(fn, wait) {
var timeout=null;
var c=function(){ clearTimeout(timeout); timeout=null; };
var t=function(fn){ timeout=setTimeout(fn,wait); };
return function() {
var context=this;
var args=arguments;
var f=function(){ fn.apply(context,args); };
timeout
? c()||t(f)
: t(c)||f();
}
}

Now, doesn't this look sharp? The symmetry is now obvious. Based on the timeout, you either clear it immediately and time out the function or you time out the clearing and execute the function immediately.

Update 26 Apr 2017: Here is an ES6 version of the function:
function debounce(fn, wait) {
let timeout=null;
const c=()=>{ clearTimeout(timeout); timeout=null; };
const t=fn=>{ timeout=setTimeout(fn,wait); };
return ()=>{
const context=this;
const args=arguments;
let f=()=>{ fn.apply(context,args); };
timeout
? c()||t(f)
: t(c)||f();
}
}

and has 0 comments
Brandon Sanderson proves again he is a brilliant writer. His Stormlight universe is not only vast and imaginative, but the characters are both compelling and well written.

Way of the Kings has some slow parts, though, and even if I kind of liked that, it is uneven in regards to its characters: some get more focus, some just a few chapters. That means that if you identify with the lead characters you will enjoy the book, but if you empathize with the lesser ones you will probably get frustrated.

I particularly enjoyed the climax. It was as it should be: the tension was rising and Sanderson just wouldn't let it go, it just kept pushing it and pushing it, filling in the motivations of the character, adding burden upon burden, making choices as difficult and as important as possible before finally allowing the release of his characters making one. Alas, the wonderful ending is followed by epilogues, several of them, which just seem boring afterwards, in comparison.

Great series, though, I recommend it highly.

I am working on this issue that I can't seem to properly be able to solve. No matter what I do I can't shake the feeling of code smell from it. It involves a chain of validations and business operations that I want to separate. The simple scenario is this: I have a web API that receives a data transfer object and needs to perform some action, but then it gets murkier and murkier.

Simply code


The first implementation is pretty obvious. I get the DTO, perform checks on it, access the database to get more info, perform more checks, do more work until everything is done. It's easy to do, clear imperative linear programming, it does the job. However I get a bad smell from the code because I have mashed together validation and business logic. I would also like to reuse the validation in other areas of the project.

Manager and Validator


So the natural improvement is to create two flows instead of one, each with their own concern: a validator will perform checks so I know an operation is valid, then the manager performs the operation. But this soon hits a snag. Let say I want to validate that a record exists from the id that I send as a property in the DTO. If I do this check in the validator, then it accesses the database. It stinks! I should probably get the record with the manager, but then I have to validate that the record was returned and that the data in it is valid in the context of my DTO. What do I do?

Manager and Validator classes


OK, so I can split the validation and operational flows in methods that will live in their respective classes. In this scenario a validator checks the record id is not malformed, a manager attempts to get the record, then the validator executes another method to check the record exists and conforms to the expectations, and so on. Problem solved, right? But no. That means I have to pass the record to the validating method, or any number of records that I need to perform the validation. For example I want to check a large chain of records that need to be validated based on conditions in the DTO. I could get the list of all records, pass them to the validator and then check if they exist, but what if the database is very large?

Manager methods that help validation and Validator


Wait, I can be smart about it. I know that essentially I want to get a list of records then validate something. Can't I just create a specific method in the manager that gets only the records involved in validation? Just do manager.GetRelatedRecords(dto.Id) to get a list of records from the database, then pass them as parameters to the validator methods. Fixed it, right? Nope. What if there is a breaking condition that is related to validation? The original linear method would just go through the steps and exit when finished. It wouldn't get a bunch of unnecessary records then fail at the first later on. And there is even a worse problem: what if while I was busy getting the records and preparing for validation the records have changed? I am basically introducing a short lived cache, with all the problems arising from it, like concurrency and invalidation.

Transactions


Hey, I can just new TransactionScope the shit out of it, right? Do steps of Validate, GetMoreData and repeat all inside a transaction. Anything fails, just rollback, what is the big deal? Well, concurrent access being slowed or deadlocks would be problematic, but more than that I am terrified of the result. I started with a linear flow in a single method and now I have two classes, each with a lot of methods which get more and more parameters as I move on: first I validate the DTO values, then a record in relation to the DTO, then two records in relation to each other and the DTO and so on. All these methods are being executed inside a transaction using a local ad-hoc memory cache to hold all the data I presume I will need. The manager methods are either using a big list of parameters themselves or repeatedly instantiating the same providers for the data they get. It's a lot more complex than the original, working design. It's also duplicated flow, as it needs to at least partially go through the data to populate what I need, then it goes through the same flow during the validation.

Context


Ha! I know! Instead of passing an increasing number of parameters to validation methods, create a single context object in which I just populate properties as I go. No more mega methods. Anything can be solved by adding another layer of indirection, correct? Well, now I have three classes, not two, with the context class practically constituting a replacement of the local scope in the original method. The validation methods are now completely obscure unless you look inside them: they all receive the DTO and the context class. Who knows which properties in the context are being used and how?

Interfaces and Flow


Ok, maybe I rushed into things. Sometimes several layers of indirection are required to solve a single problem. In this case I will create an interface with the properties and methods used by every validation method. So the name validation will look like ValidateName(MyDTO dto, INameContext nameContext), the graph validation will look like ValidateGraph(MyDTO dto, IGraphStructure graph) and my context class will implement INameContext and IGraphStructure, even when the two interfaces have common members. I will do the same with the manager class, which will implement the interfaces required to populate the specific context interfaces.

I am not kidding!


This isn't one of those "how the junior and the senior write the same code" jokes. I actually don't know what to do. The most "elegant" solution turned a working method from an API class into three classes, transactions, a dozen interfaces and, best of all, still keeping the entire logical flow in the original method. You know how when a code block becomes too complex it needs to be split into several short methods, right? Well I feel like in this case I somehow managed to split each line in the original block into several methods. I can see the Medium article title now: "If your method has one line of code, it's too long!".

Unit Tests


Any software developer worth his salt will write unit tests, especially for a code such as this. That means any large validation method will need to be split just in order to not increase unit test size exponentially. Can you imagine writing unit tests for the methods that use the huge blob-like context class?

Cry for help


So help me out here, is there an general, elegant but simple solution to API separation of concerns in relation to validation?

and has 0 comments
I am not going to keep it there for long, so try it out. It is based on Markdown CSS, by mrcoles.

and has 0 comments
Last year I wrote a blog post detailing my experience with social media after four months. This is the followup, after I've had a whole year to take advantage of these tools.

Social Media - what is it?


To me social media means the big two: Facebook and Twitter. I still have no idea what Instagram is and I don't really consider LinkedIn as social media. And Google+ is not worth mentioning. I know there are a lot of other social sites, but I ignored completely photo and video platforms - since I rarely express myself visually, and I've tried some technical platforms like StackOverflow, HackerRank or GitHub, but again didn't consider that "social media". Probably I should, since I love software development and having people to share this with would truly be a social experience for me, but I started this experiment with focus on general social interaction. Also... Slack... what the hell is that?

What I used social for


In the previous post I said that I am using Blogger to express myself, as I have done for more than a decade, and use some tool to automatically share this on Facebook and Twitter. Not surprisingly, very little people were engaged by this method of communication. It works better than RSS feeds, that's for sure, but most of the time people on social media (including myself) want to shut off their brain and read something light, not my crazy ramblings or technical posts. I've created a Facebook page for my blog, so people can use that as an entry point, if they want, but all the posts there are shares from Blogger.

I was saying that I was pleasantly surprised by Twitter and the quality of content there and less enthusiastic about Facebook. However, Twitter changed some things lately, mostly allowing videos, images and smileys (what you, young folks, call emoticons - or is it emoji?) to take less space. The effect is that there are a lot more visual opinions (let's call them that) on Twitter and thus becoming harder and harder to read. Also the amount of postings there is overwhelming. I tried to look for some tools to limit the number of tweets or organizing them somehow, but unfortunately I found none that did what I envisioned. The result is that occasionally - every week or so - I scroll through Twitter until I get tired and put links in my to read list, but most of the time I only cover a day or two of content.

I found that whenever I see something that I believe is worth sharing I put it on Facebook, rather than Twitter, mostly because I have few friends on Twitter and there is that 140 character limitation. However, most of the time I just post the link anyway and maybe say a few words. I wonder if that short circuits me thinking about the subject and then writing my opinion on it, as I am doing with this blog, but most likely I would rather not share than write so much every time, so I don't know if the occasional Facebook posts are taking away Blogger posts. I will actively try to not make it the case. I noticed, though, that people that like my Tweets are often not in my friends list, so I guess it's more general an audience. That being said, all my Facebook posts are fully public, anyway.

Speaking of Facebook, when I compile my weekly reading list I also scroll down through the Facebook wall, but even with my extension to filter posts based on own content and less images, videos and likes, I still get bored rather quickly. Sometimes the jokes are funny or the pictures interesting, but I am not really a Facebook reader. Lately I have been unfollowing people in order to keep a modicum of content curation on my wall. I was really disappointed by the events system, as well. A lot of people just mark all events they could possible go to with 'Interested' and then they never go. Also the events that appear on Facebook feel like complete bullshit most of the time anyway. The ones that I would have liked to attend either don't even appear or they are so niche that I never hear about them until it is too late.

One thing that I thought I would use Facebook for was the messenger app. And I do use it, but very rarely. In the Yahoo Messenger days I would chat a lot with people. Somehow all that became frivolous, not only for me, but other people as well. Now I see young people just getting a lot of notifications and ignoring them. So what's the point, anyway?

And speaking of... God, notifications are annoying. Everything wants to notify you of the very important thing that happened on it. It does so by blinking, beeping, animating or any other histrionic method of getting your attention. They do it incessantly until I stop caring. Notify away, I will ignore you.

What I will be using social for


I don't foresee any change in the future other than maybe using less social media altogether. I am half convinced that I should try to develop meaningful human relationships, at least as another experiment. Clearly social media does NOT connect people on a personal level at all. I've heard about these young kids that share everything they do on social media. Maybe they do, but I am not following them. To me that's another network altogether. The occasional curiosity to see what is "trending" or "popular" disgusts me every single time. The things my friends share are not truly representative of them. And if I make the first step and post some weird feeling or situation I am in, I mostly get no reaction. People avoid negative emotions unless they are manic: hate, anger, disgust take first stage while depressive thoughts, sadness or desperation are avoided. Same for positive emotions, by the way, when people are extra happy about having a child or something like that. Just mildly enjoying something and feeling good about oneself is generally ignored.

Conclusion


I am not going to commit social media suicide or anything, but I concluded that I want to know what people think, rather than what people feel, and social media is used more for the latter. Therefore my commitment to online electronic expression is not going to increase towards Facebook and Twitter. As always, I do hope I will blog more meaningful posts. Wish me luck!

and has 0 comments
This is how 2016 ends, not with a bang but a whimper. After a relatively calm Christmas followed an eventless New Year celebration. Part of me was lamenting the oldness of it all: two people alone on New Year's Eve, drinking gin tonic and campari cola and sitting at their respective electronic devices. The other part of me was happy that no one bothers me, that I don't have to pretend to enjoy loud noises and strong lights and fake emotions. Was it a nice end of year or just another win for my comfort zone?

It was a strangely quiet year for me, as well. Most of it I was in a sabbatical during which I did nothing of note and the last part was about getting hired and getting acquainted with my new place of work, which was fine. No real drama, no dead relatives I cared about, no fuss. I wrote my code, I watched my TV series, I read my tech news. It was more than quiet, it was boring.

Meanwhile, though, everybody else was going crazy: beloved celebrities died, elections went to hell just about everywhere, terror scares, immigration issues, civil wars, cyber wars, cold wars, global warming and so on and so on. Weird contrast, isn't it? Part of me laments I am growing apart from the world and people, the other part enjoys the hell out of it. Is that what growing old is? Just getting off the train and raising a middle finger?

So what about my New Year resolutions? I have none. I have some hopes, but resolutions? Nah! I am too comfortable ignoring everything that matters. If I go down that road, defining priorities, finding solutions to get what I want, getting rid of what I don't want, setting up goals, then I have to change my entire life. I have to start over. I have to make an effort, hurt people, push the drama button. Who needs that? I do have the heart of a child. I keep it in the freezer, never to be thawed.

Sometimes we want to serialize and deserialize objects that are dynamic in nature, like having different types of content based on a property. However, the strong typed nature of C# and the limitations of serializers make this frustrating.


In this post I will be discussing several things:
  • How to deserialize a JSON property that can have a value of multiple types that are not declared
  • How to serialize and deserialize a property declared as a base type in WCF
  • How to serialize it back using JSON.Net

The code can all be downloaded from GitHub. The first phase of the project can be found here.

Well, the scenario was this: I had a string in the database that was in JSON format. I would deserialize it using JSON.Net and use the resulting object. The object had a property that could be one of several types, but no special notation to describe its concrete type (like the $type notation for JSON.Net or the __type notation for DataContractSerializer). The only way I knew what type it was supposed to be was a type integer value.

My solution was to declare the property as JToken, meaning anything goes there, then deserialize it manually when I have both the JToken value and the integer type value. However, passing the object through a WCF service, which uses DataContractSerializer and which would throw the exception System.Runtime.Serialization.InvalidDataContractException: Type 'Newtonsoft.Json.Linq.JToken' is a recursive collection data contract which is not supported. Consider modifying the definition of collection 'Newtonsoft.Json.Linq.JToken' to remove references to itself..

I thought I could use two properties that pointed at the same data: one for JSON.Net, which would use JToken, and one for WCF, which would use a base type. It should have been easy, but it was not. People have tried this on the Internet and declared it impossible. Well, it's not!

Let's work with some real data. Database JSON:
{
"MyStuff": [
{
"Configuration": {
"Wheels": 2
},
"ConfigurationType": 1
},
{
"Configuration": {
"Legs": 4
},
"ConfigurationType": 2
}
]
}

Here is the code that works:
[DataContract]
[JsonObject(MemberSerialization.OptOut)]
public class Stuff
{
[DataMember]
public List<Thing> MyStuff;
}

[DataContract]
[KnownType(typeof(Vehicle))]
[KnownType(typeof(Animal))]
public class Configuration
{
}

[DataContract]
public class Vehicle : Configuration
{
[DataMember]
public int Wheels;
}

[DataContract]
public class Animal : Configuration
{
[DataMember]
public int Legs;
}

[DataContract]
public class Thing
{
private Configuration _configuration;
private JToken _jtoken;

[DataMember(Name = "ConfigurationType")]
public int ConfigurationType;

[IgnoreDataMember]
public JToken Configuration
{
get
{
return _jtoken;
}
set
{
_jtoken = value;
_configuration = null;
}
}

[JsonIgnore]
[DataMember(Name = "Configuration")]
public Configuration ConfigurationObject
{
get
{
if (_configuration == null)
{
switch (ConfigurationType)
{
case 1: _configuration = Configuration.ToObject<Vehicle>(); break;
case 2: _configuration = Configuration.ToObject<Animal>(); break;
}
}
return _configuration;
}
set
{
_configuration = value;
_jtoken = JRaw.FromObject(value);
}
}
}

public class ThingContractResolver : DefaultContractResolver
{
protected override JsonProperty CreateProperty(System.Reflection.MemberInfo member,
Newtonsoft.Json.MemberSerialization memberSerialization)
{
var prop = base.CreateProperty(member, memberSerialization);
if (member.Name == "Configuration") prop.Ignored = false;
return prop;
}
}

So now this is what happens:
  • DataContractSerializer ignores the JToken property, and doesn't throw an exception
  • Instead it takes ConfigurationObject and serializes/deserializes it as "Configuration", thanks to the KnownTypeAttributes decorating the Configuration class and the DataMemberAttribute that sets the property's serialization name (in WCF only)
  • JSON.Net ignores DataContract as much as possible (JsonObject(MemberSerialization.OptOut)) and both properties, but then un-ignores the Configuration property when we use the ThingContractResolver
  • DataContractSerializer will add a property called __type to the resulting JSON, so that it knows how to deserialize it.
  • In order for this to work, the JSON deserialization of the original text needs to be done with JSON.Net (no __type property) and the JSON coming to the WCF service to be correctly decorated with __type when it comes to the server to be saved in the database

If you need to remove the __type property from the JSON, try the solution here: JSON.NET how to remove nodes. I didn't actually need to do it.

Of course, it would have been simpler to just replace the default serializer of the WCF service to Json.Net, but I didn't want to introduce other problems to an already complex system.


A More General Solution


To summarize, it would have been grand if I could have used IgnoreDataMemberAttribute and JSON.Net would just ignore it. To do that, I could use another ContractResolver:
public class JsonWCFContractResolver : DefaultContractResolver
{
protected override JsonProperty CreateProperty(System.Reflection.MemberInfo member,
Newtonsoft.Json.MemberSerialization memberSerialization)
{
var prop = base.CreateProperty(member, memberSerialization);
var hasIgnoreDataMember = member.IsDefined(typeof(IgnoreDataMemberAttribute), false);
var hasJsonIgnore = member.IsDefined(typeof(JsonIgnoreAttribute), false);
if (hasIgnoreDataMember && !hasJsonIgnore)
{
prop.Ignored = false;
}
return prop;
}
}
This class now automatically un-ignores properties declared with IgnoreDataMember that are also not declared with JsonIgnore. You might want to create your own custom attribute so that you don't modify JSON.Net's default behavior.

Now, while I thought it would be easy to implement a JsonConverter that works just like the default one, only it uses this contract resolver by default, it was actually a pain in the ass, mainly because there is no default JsonConverter. Instead, I declared the default contract resolver in a static constructor like this:
JsonConvert.DefaultSettings = () => new JsonSerializerSettings
{
ContractResolver=new JsonWCFContractResolver()
};

Now the JSON.Net operations don't need an explicitly declared contract resolver. Updated source code on GitHub


Links that helped

Configure JSON.NET to ignore DataContract/DataMember attributes
replace WCF built-in JavascriptSerializer with Newtonsoft Json.Net json serializer
Cannot return JToken from WCF Web Service
Serialize and deserialize part of JSON object as string with WCF