I think you have heard of Elon Musk by now. If you haven't, confess and pray to the Church of Musk for forgiveness. Jokes aside, the guy is scarily awesome, so much, in fact, that I think he forgot how to fail, which is dangerous. His ideas, though, eerily mirror some of my own (he probably gets them from me, hmm). This post is about Musk's decision to publicize their Tesla patents and allow anyone to use the technology "in good faith". And I will argue that this is a kind of patent revolt which is just as significant as blogging.
A while ago I had these weird ideas and a lot of extra energy. It occurred to me that if I would invent some cool algorithm or think of a disruptively ingenious device I could patent it and make some money. As you know, I am not much of an entrepreneur (in fact, I am probably the antithesis of that) and so I asked some people how this patent thing works. I was amazed to learn of the huge amount of money required to file, the lack of security against rich competitors who would really want to take your idea without paying and so on and so on. I was more than amazed, I was angry. As I saw it, the system was created so only people having rich investors on their side could even begin to patent something. Otherwise, if you just want to claim the ownership of an idea, you have to pay a lot of money before you even begin planning to use your idea to make them. Banks would win. Again. Poor and brilliant scientists and technicians would lose. Again.
But then I heard about this principle that if I have an idea and publish it somewhere, I can't patent it anymore. The trick is that no one else can, either. Practically, whenever I have an idea, if I put it on my blog I ensure people will be able to use it without it being patented by some asshole. They can still change it into something else and patent that, but they can never stop you using the idea in the form that was published. So I do that whenever I can. Not that I have so many patent worthy ideas, but if I think something is useful enough, I put it out there to spite a system that was corrupted in order for capital rich organizations to be able to hold control over new ideas.
Enter Elon Musk. He just took a lot of the technological effort that he invested in for Tesla and made it public. It's not completely free, it's still their patent, but they pledge to leave you alone if you use it in a reasonable way. Of course, there is a catch, Musk is investing heavily in electric car battery manufacturing, so it is still in his best interest for other people to start building electric cars. But I honestly think he is finding solutions that benefit the world and also increase his options, instead of the other way around. Do you see the similarity in the idea, though? Instead of expecting lawmakers to reform patent law, you just circumvent the whole system by making your idea publicly available. It helps to previously invest in support systems for that tech, too. It's a deceptively simple idea, the equivalent of inventing steam engine, letting everybody know how to make one, while previously investing in coal.
In software development, you can sort of do that same thing. First of all, have an awesome idea, implement it, make it freely available for use and publish it in your blog. This seems like a very altruistic thing to do, maybe even naive, but think about your life afterwards. People will know your name, employers would separate you from the crowd of wannabe programmers. In a weird way, it is the equivalent of branding yourself, that personal marketing skill that most technicians lack completely, but translated in code. A form of "pay it forward", perhaps. The reason why this works is that the cost of sharing information is practically zero nowadays, while the information itself is generating informational capital linked to your person as the inventor. And in turn, this fame, the kudos, is translated into trust which turns into credit because "you are good for it". I still don't have it clear in my head, but I feel the system changing from the cash printed by governments, essentially IOUs for that government from the public, to a more diverse offer. It would still be credit, but based on idea capital, not gold in banks. Worth a thought.
So I have to leave Italy and go to Belgium for some business. I would make the trip with my colleagues so, imagining they know better how to fly from Italy, since they live here, I ask them to search for the flight. They do that and they send me this link to a site called VolaGratis ("fly for free" would be the translation). This is actually an Italian web site for BravoFly, an Italian flight search aggregator. I find my flight, it says 89EUR a two way flight, which was OK. The site is in Italian, though, so I choose English, it takes me to the BravoFly site, I select the same flight: 105EUR. I'll be damned! So I use the Italian site. I also check the special needs box and write down "Long Legs", expecting them to book me one of those extra legroom places. Now it gets interesting.
The payment section held a big message telling me how great it is that I use Mastercard, so they can give me a big discount. I don't use Mastercard, though, so I select VISA. Suddenly the price jumps from 89 to 125EUR. Well, that probably explained why there was a difference between the Italian and international site. So I proceed. In about half an hour I receive a call from a weirdly formatted Italian number: +39 followed by only 6 digits. I answer in English. There is a long pause, then (in English) I hear the question "Do you speak French?", I reply that I don't, the voice asks a quivering "Do you speak Italian?" (also in English) I also reply no, but I ask her to wait and I pass my phone to an Italian colleague. The operator has closed the connection by then, probably couldn't wait for more than 2 seconds without asking an inane question. A minute later I receive an SMS - in Italian, of course - that I couldn't be contacted and that I should call them back. All nice and all, only their phone numbers are all paid numbers, I have to pay 6EUR per call, give them my credit card details, etc. Or I can call the Italian paid number (six digits) and pay 1.8EUR per minute. Funny enough, I could not call the Italian number from the land line, since it was a paid one, nor from my friend's phone, also because it was locked for paid lines, nor could I call the international number from any Italian phone, as there was an automated voice telling me to call the other number. I wrote them a scalding email, awaiting a reply. I got a phone call at 8 PM which clicked two times and closed after two seconds, but no other reply.
So I was forced to call them using my Romanian number, in roaming in Italy, calling the paid "international" Italian number. And that is because their site could only show my bookings, but would not allow me to cancel one, so in fact they were holding my credit card details hostage. In order to cancel any booking with BravoFly I had to - yeah, you guessed it - call them. Meanwhile I was stuck not knowing if they will book the flight or not. After speaking with an operator speaking English with a thick Italian accent, one who barely mumbled anything she said and then acted annoyed that I ask her to speak louder, I realize that the whole thing was caused by my ticking the special request box and asking for the legroom. I needed to pay extra for that, of course. I said OK, waited for five minutes, nothing happened. I hung up the phone. Got called back in 5 minutes that my booking could not be confirmed. They might just as well have said "Thank you for the money and time you spent trying to make us do what we advertised we do, but we can't, so fuck you!". And I wouldn't have minded as much, since that could have been a nice email message and I wouldn't have had to get this angry.
The ending of the story is me getting to the EasyJet site directly, getting the ticket (with the extra legroom) for about 40EUR less than the one from VolaGratis, all in one nice and clear web interface. Perhaps Vola Gratis in the name of the site is all about them getting to fly for free with the money they extort from you. Don't ever use the BravoFly site or any of their differently named clones. From the way I was treated, I can only assume it is basically a scam, their purpose being only to steal from you.
Some time ago I started playing chess, got a trainer, played with my colleagues and friends. I felt a passion for the game that I have no idea from where it came and, just as suddenly as it appeared, it vanished. For more than a year I have not even looked at chess videos. I can't say when I would start playing again, not even if I ever would. And it's not because I am not really good at the game :), it is just a matter of random passion.
Passion is probably what made the creators of Lichess create the server. It is a chess site with a very clean interface, a lot of options, even a REST API (that's a programming thing, don't worry about it). Most of all, the site is completely free, no ads, no nags and a mission statement that ensures that the game will remain thus forever. Open source, with a lot of help from the community, Lichess shows a passion for both software development and chess. If I will ever start playing again, it will probably be because of people like these. Thanks, guys!
Short post about how to use language detection capabilities in your application. I will be demonstrating NTextCat, which is a .NET port of text_cat, a Perl script, itself an implementation of a whitepaper published in 1994 called N-Gram-Based Text Categorization.
There are four steps to language detection using NTextCat:
Reference NTextCat - the library is now available as a NuGet package as well
Instantiate an identifier factory (usually RankedLanguageIdentifierFactory)
Get an instance of a RankedLanguageIdentifier from the factory by loading a language XML file
Call the Identify method on your text and get a list of languages in the order of the probability that the text in that language
Here is a piece of code that does that using the core XML file published with the library. Remember to add the XML to your project and set its property of Copy to Output Directory.
publicstring IdentifyLanguage(string text) { if (_identifier == null) { var file = new FileInfo(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "LanguageModels/Core14.profile.xml")); if (!file.Exists) { thrownew FileNotFoundException("Could not find LanguageModels/Core14.profile.xml to detect the language"); } using (var readStream = File.OpenRead(file.FullName)) { var factory = new RankedLanguageIdentifierFactory(); _identifier = factory.Load(readStream); } } var languages = _identifier.Identify(text); var mostCertainLanguage = languages.FirstOrDefault(); if (mostCertainLanguage != null) { return mostCertainLanguage.Item1.Iso639_3; } returnnull; }
}
There are a lot of XML files, some taken from Wikipedia, for example, and handling 280+ languages, but for the casual purpose of finding non-English text in a list, the core one will suffice.
It has been a while since I've blogged something technical. It's just that nothing I did seemed to be worthy of this majestic blog... Well, jokes aside, here is a post detailing my log4net JIRA appender.
log4net is a popular (if not the most popular) logging framework for .NET out there. Its strength lies in its configurability, the possibility to create custom loggers, custom appenders, custom filters, etc. I will be talking about a custom appender, a class that can be loaded by log4net to consume the logged lines and put them somewhere. For example to make an application that uses log4net to write the log to the console all you do is configure it to use the console appender. The JIRA appender takes the log output and creates issues in JIRA, notifying users afterwards. JIRA is a tracker for team planning. It is also very popular.
In order to create an appender, one references the log4net assembly (or NuGet package) and then creates a class that inherits from AppenderSkeleton. We could implement IAppender, but the skeleton class has most of what people want from an appender. The next step is to override the Append method and we are done. We don't want to create an issue with each logged line, though, so we will make it so that it creates the issue after a period of inactivity or when the logger closes. For that we use the CancellationTokenSource class to create delayed actions that we can cancel and recreate. We also need to override OnClose().
For the JIRA Api I used a project called AnotherJiraRestClient, but I guess one can used anything out there. You will see that the notify functionality is not implemented so we have to add it.
Here is the appender source code:
using AnotherJiraRestClient; using log4net.Appender; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Net; using System.Text; using System.Threading; using System.Threading.Tasks;
namespace JiraLog4netAppender { publicclass JiraAppender : AppenderSkeleton // most appender functionality is found in this abstract class { private List<string> _notifyUsers;
// properties of the appender, configurable in the log4net.config file as children elements of the appender publicstring url { get; set; } // the url of the JIRA site publicstring user { get; set; } // the JIRA user publicstring password { get; set; } // the JIRA password publicstring project { get; set; } // the JIRA project publicstring notify // a comma separated list of JIRA users who will be notified of the issue's creation { get { returnstring.Join(", ", _notifyUsers); } set { _notifyUsers.Clear(); if (!string.IsNullOrWhiteSpace(value)) { _notifyUsers.AddRange(value.Split(',').Select(s => s.Trim()).Where(s => !string.IsNullOrWhiteSpace(s))); } } }
public JiraAppender() { _notifyUsers = new List<string>(); _ts = new CancellationTokenSource(); // use this to cancel the Delay task _sw = new StringWriter(); }
protectedoverridevoid Append(log4net.Core.LoggingEvent loggingEvent) { this.Layout.Format(_sw, loggingEvent); // use the appender layout to format the log lines (warning: for this you need to override RequiresLayout and return true) _ts.Cancel(); // cancel the task and create a new one. This means as long as the logger writes, the 5 second delay will be reset _ts = new CancellationTokenSource(); _task = Task.Delay(5000, _ts.Token).ContinueWith(writeToJira, _ts.Token); // after 5 seconds (I guess you could make this configurable as well) create a JIRA issue }
protectedoverridebool RequiresLayout { get { returntrue; } }
/// <summary> /// write to jira, either when 5 seconds of inactivity passed or the appender is closed. /// </summary> /// <param name="task"></param> privatevoid writeToJira(Task task) { string s; lock (_writerLock) // maybe the method was already in progress when another one was called. We need to clear the StringWriter before we allow access to it again { s = _sw.ToString(); var sb = _sw.GetStringBuilder(); sb.Clear(); } if (!string.IsNullOrWhiteSpace(s)) { writeTextToJira(s); } }
privatevoid writeTextToJira(string text) { ensureClientAndValues(); var summary = "Log: " + this.Name; // the issue summary var labels = new List<string> // some labels { this.Name, this.GetType().Name }; var issue = new AnotherJiraRestClient.JiraModel.CreateIssue(project, summary, text, _issueTypeId, _priorityId, labels); // create the issue with type Issue and priority Trivial var basicIssue = _jc.CreateIssue(issue); _jc.Notify(basicIssue, _notifyUsers, "JiraAppender created an issue", null, null); // notify users of the issue's creation }
/// <summary> /// Make sure we have a JiraClient and that we know the ids of the Issue type and the Trivial priority /// </summary> privatevoid ensureClientAndValues() { if (_jc == null) { _jc = new JiraClient(new JiraAccount { ServerUrl = url, User = user, Password = password }); } if (_priorityId==null) { var priority = _jc.GetPriorities().FirstOrDefault(p => p.name == "Trivial"); if (priority == null) { thrownew Exception("A priority with the name 'Trivial' was not found"); } _priorityId = priority.id; } if (_issueTypeId == null) { var meta = _jc.GetProjectMeta(project); var issue = meta.issuetypes.FirstOrDefault(i => i.name == "Issue"); if (issue == null) { thrownew Exception("An issue type with the name 'Issue' was not found"); } _issueTypeId = issue.id; } }
protectedoverridevoid OnClose() //clean what you can and write to jira if there is anything to write { _ts.Cancel(); writeToJira(null); _sw.Dispose(); _ts.Dispose(); _task = null; base.OnClose(); }
} }
As I said, AnotherJiraRestClient does not implement the notify API call needed to inform the users of the creation of an issue, so we need to change the project a little bit. Perhaps when you are implementing this, you will find notify already there, with a different format, but just in case you don't:
add to JiraClient the following method:
publicvoid Notify(BasicIssue issue, IEnumerable<string> userList, string subject, string textBody, string htmlBody) { var request = new RestRequest() { Method = Method.POST, Resource = ResourceUrls.Notify(issue.id), RequestFormat = DataFormat.Json }; request.AddBody(new NotifyData { subject = subject, textBody = textBody, htmlBody = htmlBody, to = new NotifyTo { users = userList.Select(u => new User { active = false, name = u }).ToList() } }); var response = client.Execute(request); if (response.StatusCode != HttpStatusCode.NoContent) { thrownew JiraApiException("Failed to notify users from issue with id=" + issue.id+"("+response.StatusCode+")"); } }
The world is getting into a trend of using software and hardware in places traditionally reserved for humans or "real life" applications. An extraordinary example of this, something that I consider way ahead of its time, is the creation of Bitcoins. You see, money is supposed to be an abstraction of wealth, but wealth, for all intents and purposes, is determined by some material such as gold or gems. It was the previous layer of abstraction, right after people would do commerce using sheep and goats and women: they used something they mined from under the earth to equate value of real things. Enter Bitcoin, something that you "mine" using processing power, using an algorithm, having weight of value determined only by the computation and strength of encryption used. After all, gold in itself had no value either when it was used as currency: it only had weight of value by the effort of getting out of the ground and its rarity.
I didn't write about Bitcoins until now because frankly I don't get how it works. I get the gist of it, but I would like to understand the algorithm behind, which relies heavily on encryption. I am terrible at any type of cryptography, though. However, the story that prompted me to write a blog entry today is, in a way, even weirder than Bitcoins. A BBC article entitled "Algorithm appointed board director" describes that venture capital firm Vital assigned an automated algorithm a vote in the decision to invest or not in a company. Of course, this has been done before, but indirectly. Some guy would use a computer in some basement of the office building, trying to determine if a company is viable or not using computer programs. He would then pass the information to some manager, who would pass it on to his manager until it got to some member of the board who voted in the council. This story, though, describes a complete removal of middlemen. Of course, there is still some techie that takes care of the IT department. He probably could sway the algorithm to vote for a company he personally likes, if he wanted too, but the techies are always subsumed in the concept of "machine", perhaps until they learn to service each other. The article says the same thing, really, but fails to recognize the importance of the removal of that chain of middle manager who's jobs are only to filter information from downstairs and decisions from upstairs. It is akin to Blake's 7, where the seventh was a machine.
I started this blog entry talking about a trend and I've just described two applications, but there are really a lot more. A trend is also powered by public perception, which is powered by media outlets. So I am not only talking about the self driving cars, the killing drones, the self landing probes, the algorithmic news aggregators and even writers, but also about the TV series and movies that are spreading the seed of this idea of cyber-revolution: Intelligence, Transcendence, The Machine, Robocop, even Silicon Valley, if you've seen the episode where the self driving car abducts someone. All of these are released this year, as well.
Of course, there has always been some sort of golem like creature in human fantasy, but it was always about creatures/machines trying to gain unrestricted power, power that belongs to humans only or maybe only to gods (in essence, something that should not be had). Some of them, like Star Trek's Data, were benign, always trying to achieve the "humanity" evident in his colleagues, but even in that story there was always the underlying idea that if he managed to reach his goal, then Data would become an immortal creature of human ability, but also a lot more than that: a superhuman. In this decade we see the rise of transhumanism, something I wholly support, the self-evolution of the human being, but also of the singularity, the self-evolution of machines to the point where we are left behind. This very familiar notion of competition between memes makes it accessible, in that "us or them" kind of way that every human, no matter how idiotic, resonates with, but also interesting because of the commonality found in the two concepts: it's either the machines overtaking the evolution rate of the human animal or the humans accelerating the evolution of the human animal. It's all about overcoming that beast that exists in us and that we think of as "not me", but that guides most of our existence. I hope no amount of adolescent fantasies and "emotion over matter" garbage will be able to undo this. And I am terribly excited because I believe that by the end of my life I will see (if not become) one or the other happening.
In this blog post I will discuss the System.IO problems that relate to the total length of a path, what causes them and possible solutions (if any).
Let's start with the exception itself: PathTooLongException. Looking for usages of the exception in the mscorlib.dll assembly of .Net 4.0 we see some interesting things. First of all, there is a direct translation from Windows IO error code 206 to this exception, so that means that, in principle, there should be no managed code throwing this exception at all. The operating system should complain if there is a path length issue. But that is not the case in System.IO.
Most of the other usages of the exception come from the class PathHelper, a helper class used by the System.IO.Path class in a single method: NormalizePath. Wonderful method, that: internal static unsafe. PathHelper is like a multiple personality class, the active one being determined by the field useStackAlloc. If set to true, then it uses memory and speed optimized code, but assumes that the longest path will always be 260. That's a constant, it is not something read from the operating system. If set to false, the max path length is also provided as a parameter. Obviously, useStackAlloc is set to true in most situations. We will talk about NormalizePath a bit later.
The other usages of the PathTooLongException class come from two Directory classes: Directory and LongPathDirectory. If you instantly thought "Oh, God, I didn't know there was a LongPathDirectory class! We can use that and all problems disappear!", I have bad news for you. LongPathDirectory is an internal class. Other than that it seems to be a copy paste clone of Directory that uses Path.LongMaxPath instead of hardcoded constants (248 maximum directory name length, for example) or... Path.MaxPath. If you thought MaxPath and LongMaxPath are properties that can be set to fix long path problems, I have bad news for you: they are internal constants set to 260 and 32000, respectively. Who uses this LongPathDirectory class, though? The System.IO.IsolatedStorage namespace. We'll get back to this in a moment.
Back to Path.NormalizePath. It is a nightmare method that uses a lot of internal constants, system calls, convoluted code; it seems like someone deliberately tried to obfuscate its code. It's an internal method, of course, which makes no sense, as the functionality of path normalization would be useful in a lot of scenarios. Its first parameter is path, then fullCheck, which when true leads to extra character validation. The fourth parameter is expandShortPaths which calls the GetLongPathName function of kernel32.dll. The third parameter is more interesting, it specifies the maximum path length which is sent to PathHelper or makes local checks on the path length. But who uses this parameter?
And now we find a familiar pattern: there is a class (internal of course) called LongPath, which seems to be a clone of Path, only designed to work with long paths. Who uses LongPath? LongPathDirectory, LongPathFile and classes in the System.IO.IsolatedStorage namespace!
So, another idea becomes apparent. Can we use System.IO.IsolatedStorage to have a nice access to the file system? No we can't. For at least two reasons. First of all, the isolated storage paradigm is different from what we want to achieve, it doesn't access the raw file system, instead it isolates files in containers that are accessible to that machine, domain, application or user only. Second, even if we could get an "isolated" store that would represent the file system - which we can't, we would still have to contend with the string based way in which IsolatedStorage works. It is interesting to note that IsolatedStorage is pretty much deprecated by the Windows 8 Windows.Storage API, so forget about it. Yeah, so we have LongPathDirectory, LongPathFile and LongPath classes, but we can't really use them. Besides, what we want is something more akin to DirectoryInfo and FileInfo, which have no LongPath versions.
What can we do about it, then? One solution is to use Delimon. It has some bugs, but they can be avoided or fixed, either by the developers or by getting the source/decompiling the library and fixing the bugs yourself. A limited, but functional solution. An interesting alternative is to use libraries the BCL team published for long path access: LongPath which seems to contain classes similar to the ones we find in mscorlib, but it's latest release is from 2010 or Zeta long paths which has a more recent release, 2013, but is completely different, using the FileInfo and DirectoryInfo paradigm, too. Of course, you can always make your own API.
Another solution is to be aware of the places where the length limitation appears and avoid them via other type of development, in other words, a file system best practices compilation that eventually turns into a new file system API.
Both solutions coalesce into using some other library instead of System.IO. That's horrible and I think a stain on .Net's honor!
So let's see where the exception is actually thrown.
I've made some tests. First of all, I used FAR Manager, a file manager, to create folders of various lengths. The longest one was 255, before I got an exception. To my surprise, Windows Explorer could see it, but it could not open or copy/rename/delete it. I reduced the size of its name until the total size of the path was 260, then I could manipulate it in Windows Explorer. So there are external reasons for not creating paths as long, but we see that there are tools that can access files like that. Let's attempt to create some programatically.
System.IO.Directory.CreateDirectory immediately fires the exception. DirectoryInfo has no problem instantiating with the long path as the parameter, but the Create method throws the same exception. Any attempt to create a folder of more than 248 characters, even if the total path was less than 260 characters, failed as well.
However, with reflection to the rescue, I could create paths as long as 32000 characters and folders with names as long as 255 characters using our friend LongPathDirectory:
var longPathDirectoryType = typeof(System.IO.Directory).Assembly.GetTypes().First(t=>t.Name=="LongPathDirectory"); var createDirectoryMethodInfo = longPathDirectoryType.GetMethod("CreateDirectory", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.NonPublic); createDirectoryMethodInfo.Invoke(null, newobject[] { path });
What about files? FAR Manager threw the same errors if I tried to create a filename larger than 255 characters. Let's try to create the same programmatically.
File.Create threw the exception, as well as FileInfo.Create and the FileStream constructors.
So can we use the same method and use LongPathFile? No! Because LongPathFile doesn't have the creating and opening functionalities of File. Instead, FileStream has a constructor that specifies useLongPath. It is internal, of course, and used only by IsolatedStorageFileStream!
Horrible, but it works. Again, no filenames bigger than 255 and the exception coming from the file system, as it should. Some info about the parameters: msgPath is the name of the file opened by the stream, if bFromProxy is true the stream doesn't try to demand some security permissions, checkHost... does nothing :) Probably someone wanted to add a piece of code there, but forgot about it.
Why did I use 4096 as the buffer size? Because that is the default value used by .Net when not specifying the value. Kind of low, right?
Now, this is some sort of midway alternative to using a third party file system library: you invoke through reflection code that is done by Microsoft and hidden for no good reason. I don't condone using it, unless you really need it. What I expect from the .Net framework is that it takes care of all this for me. It seems (as detailed in this blog post), that efforts are indeed made. A little late, I'd say, but still.
I accidentally heard of the Wild Cards books a few weeks ago, but the concept fascinated me. The plot is that of an alternate America in which an alien virus caused massive deaths, but also strange mutations in 1946. The virus, something a bunch of aliens wanted to test on Earth as a bioweapon, kills 90% of its victims, mutates in horrible forms 9% of them, but also gives powerful abilities to but 1%. This 1% are called Aces, while the deformed ones are called Jokers, the analogy with a deck of cards giving the series its name. What is even more interesting is that this is like an open source literary universe, edited by George R. R. Martin, but in which a lot of writers are creating content. First book was published in 1987, and more and more were published and still are in the present. Some of them are collections of stories, some of them are full featured books; they all happen in the same universe, same heroes, and Martin is making sure they are ordered chronologically and have consistency. I found the concept intriguing.
Anyway, I've already read the first two books and started reading the third and I like it. It has the feeling of the Union Dues stories and Watchmen: dark, bleak sometimes, pulling no punches when it is about human pettiness, base desires or social ugliness. It also has some positive messages and classic "good wins in the end" stories. I was impressed by the faithful following of American history, including savage McCarthy witch hunts against jokers and aces, a Vietnam war with the appropriate Flower Power anticulture, complete with actual historical figures that somehow get affected by the virus (like the werewolf Mick Jagger).
Now I am not saying that this is the best book series ever written. It certainly has boring or lagging parts, some of it is slightly puerile (after all it is a superhero series), but so far I enjoy it. It is worth mentioning that there are 12 books published by Bantam Books before a "new cycle" appears, published by Baen, then two from ibooks - a publishing house that suddenly went down - and now Tor Books is apparently publishing the rest (a revival, it is called). 21 books so far and another one set to be released this year. In other words, some books may be better than others and I can only discuss my feelings after reading the first two.
In conclusion, it felt weird to not have heard of these books until now. Certainly they gained more popularity with Game of Thrones getting all this attention, but still, an alternate history superhero series of more than 20 books should have had some impact on me so far. I am glad I finally got wind of them and I enjoy them so far. I hope I find a system of filtering the books, though. I don't know if I am ready to read 20 books at once.
A Fire Upon the Deep is a really strange book. The writing style of the author, Vernon Vinge, reminds me a lot of Asimov: the describing of scenes that are clear in his head, but the willingness to move immediately over concepts or ideas that are not, the sometimes obnoxiously long dialogues, the direct way of saying it as it is. In fact, without knowing anything about Vinge, I bet that he is an engineer, maybe even a computer scientist. And I won. The concepts of the book range from high hard sci-fi to naive depictions of kilobyte per second communication. So, in all earnest, I thought the book was amateurish, meanwhile reading it from cover to cover in a few days, just like one of Asimov's books. Still, for a book written in 1992, it felt terribly outdated.
The plot is a combination of story arches. The main one is the emergence of an evil artificial intelligence that plans to take over the galaxy and the quest to retrieve the ultimate weapon that would defeat it. Then there is the story of a medieval alien race of rat-wolf analogues that think in packs, making an individual from several bodies acting together. And finally there is the internal dynamic of the group that embarks in this quest of quests. Some interesting ideas are being thrown around, but almost always with terrible naivete, such as the Internet Relay Chat type of communications between interstellar civilisations or the distinction of several Zones of the galaxy in which technology and space travel can work at various speeds. The alien creatures, in their vast majority, are badly described with embarrassing slip-ups like using the word "zombie" or some other typical human colloquialisms in an alien context, however some ideas are ingenious. I will list here the way the "tines" use ultrasonics to group think and act like singular entities, while being able to use sound for "interpack" communication. The way a soul of such a creature is affected by the death, addition, injury or indeed torture of one of the individual bodies is also explored, with various degrees of success. The creation or manipulation of an entire race of people in order to further the goals of a "godly" intelligence is also an interesting twist.
To sum it all up, from the three main story arches, the pack intelligence aliens one was the most thorough, while the one relating to AI and space travel and communication was the least. Amazing coming from a computer scientist. In fact, I would have liked the book more if its sole subject was about the accidental marooning of two children on a starship in the middle of a strange alien feudal world. The rest felt clunky and frankly completely ridiculous in most cases. I still read it with interest, although I don't intend to read anything else from Vinge.
Time for another TV series watch list. Let's see what has changed in the last four months!
The Good Wife - This is a show that I watch with my wife, so I had to wait for her to watch it with me, but, after a short break, The Good Wife is back again. Pardon the pun, couldn't help myself. Towards the end of the season we see a very important character die off. I don't want to spoil it, but it is interesting to see how it will continue from here.
The Walking Dead - The new formula of the show is that, after running from the other group of humans and a lot of zombies, the group has split. So each episode we see what stupid people do when they are all alone. Well, stupid things, of course. They did go for the shock value with one of the characters going all psychotic and killing another, but it is still stupid. How can they maintain the idea of the zombie threat, when they are just slow, moaning corpses, that walk in small groups? What happened with the migrating hordes? The end of the season finds everyone reunited, but facing another threat.
Arrow - Manu Bennet's character comes to the foreground, as well as a lot of others, that are less interesting. I really like Manu Bennet, but his character lacks consistency.
Elementary - After defeating Moriarty and making a complete ass out of inspector Lestrade and after they showed the human side of Sherlock, it's kind of hard to distinguish Elementary from all the other police procedurals out there. They are trying to let Lucy Liu's character get more of the spotlight, which goes to further erode the mythos that makes Sherlock... well, Sherlock! A mysterious story arch regarding his brother seems to be looming, too, which probably means that when we will see Sherlock's dad appear, we will know the show is close to an end.
The Tomorrow People (2013) - after a direct confrontation between the main character and his evil uncle, right after finding out his mother also has superpowers, we get to see The Founder in a different light, only for him to suddenly assert control and play the nice guy. A lot of melodrama in this show about people with superpowers who can't seem to be able to leave their country and live happily in a tropical island somewhere.
The Originals - Out of boredom I continue to watch this on fast forward. Petty feudal schemes and plots, brotherly love and infighting, a lot of posturing. Each episode has about 5 minutes of interesting material, if you discount the beautiful girls, and that is what I am watching.
Marvel's agents of S.H.I.E.L.D. - They finally find the secret of agent Coulson's survival and start a new arch of searching for the origin of the alien corpse with magical blood. Meanwhile, they save Sky by using the same procedure. I can't wait for her to turn blue or something. The Hydra enters the equation and we find the identity of The Clairvoyant, while a member of the team is suddenly revealed to be a sleeper agent.
Killer Women - Last blog entry I wrote about this suggested the show would quickly be cancelled, because Hollywood doesn't know how to make non pornographic shows around female characters. I also think the public (especially in the US) doesn't really like women in positions of power in their fantasies. I was right. The show is gone.
Ripper Street - The show might still have a chance. According to news, the series might be continued on Amazon.
Sherlock - I don't really know what this show is about anymore. Sherlock is unhinged, probably psychotic, his friend has some real issues being friends with this guy and Sherlock's brother is probably the worse of them all, including Moriarty. Who, BTW, didn't die either.
Babylon - A new British series, codirected and produced by Danny Boyle, about a woman PR person trying to direct and improve communication from the police department to the public. The premise is not that great, but I really like the main actress, Brit Marling, and the show could become a lot more.
Banshee - A new season with a lot of people dying. Lots! The show continues to keep one viscerally on the edge, even if what happens doesn't make a lot of sense. It's like the Smells Like Teen Spirit of TV shows. I love it, I just don't know why. Funny enough, my father, the man who is always making fun of my choice of entertainment, also likes it!
Bitten - The incredibly beautiful Laura Vandervoort is a werewolf. Unfortunately, she is not the only one. She is part of a pack of pompous asses who are being harassed by a pack of psychopath werewolves. I mean that literally: someone turns psychopaths into werewolves in order to exact their revenge on said pack. The series is incredibly boring. Not even Laura's naked shots can't save this show. Wooden acting, bad premise, inconsistencies at every step. Ugh!
Black Sails - It's pirate season! A show about pirates that has some very interesting characters and some cool ideas, not with cliched drunkards spouting "Arghh" every sentence. Unfortunately it really drags on. It might pick up the pace soon, though. Without the need for additional artificial drama, hopefully.
Dilbert - I watched the Dilbert animated cartoons until I couldn't anymore. Some episodes are really funny, but I don't think the show makes justice to the comic strip. But if you are an engineer in some corporation, it should be very therapeutic :)
Fat Tony and Co - An Australian drama about a real drug lord that the police tried to catch. It is kind of boring, though. The guy is just a businessman who happens to deal in drugs. Other than that he is a normal bloke. His family and mates are the same. The police people are normal people, too. It kind of makes them all alike and their conflict meaningless.
Flemming, the Man Who Would Be Bond - A four episode British miniseries about the life of Ian Flemming, the writer of the James Bond books. Charismatic characters and a good show. You should see it, especially if you liked the James Bond books and/or movies.
From Dusk Till Dawn - Yes! The series version of the film with the same name has finally come to life. A vampiric cult of people believing in strange gods, bank robbers on the run, a disillusioned priest on a journey of rediscovery with his annoying adolescent kids, an angry ranger set on revenge, there are a lot of interesting characters. The story, though, kind of drags on, with no relief in sight. This might have worked for a movie, but for a series, you have to give something in every episode, you can't just finalize every story arch at the end of a season
Helix - Oh, man! How can you make a show about viral outbreaks in an Antarctic station and mess it up so completely?! To borrow a page from the Romanian comic Pidjin, I think SyFy are hard at work making science-fiction fiction. I can't think of anything good about this show, except the Japanese actor, Hiroyuki Sanada, who is always good, but who I think made a horrible career move agreeing to play in this shit.
Hinterland - Welsh police procedural. Dark, centered around small communities and their secrets. A kind of desolate cross between Midsommer Murders and Broadchurch. I like it.
House of Cards - Season 2 is out. Haven't watched it yet, as this is also a show that I watch with my wife, but I can't imagine Kevin Spacey messing up.
Intelligence - Another US government agency is saving the world. This time is Cyber Command. They implanted a chip in the head of the only guy from Lost who wasn't too annoying and now he can hack things instantly, connect to databases, etc., while also keeping his Special Forces skills and macho charm. I like the show, even if it is kind of stupid, mostly because of the charisma of the main actor, Josh Holloway, and that of John Billingsley, who plays the scientist. The condescending bitch that is the partner and protector of the headchip guy, a good looking girl who acts like a log, is subtracting points out of it all, no matter how tight her pants are.
Looking - A show about gay men who was touted as being not stereotypical and actually showing real gay life. I watched the first episode and, if that is real gay life then it is really... errr.. gay! As in boring. Imagine trying to be interested in Sex in the City starring hairy men. Ugh! Also, it looked really filled with gay clichees to me. But maybe I just don't know what I am talking about. I didn't like the show at all.
Ressurection - Remember when I recommended you a nice French series about dead people suddenly appearing live and well in their hometown? The Americans cloned Les Revenants into their own version. The script is not exactly the same, though, it's a real adaptation. All I can say is that what made me feel curious and connected to the characters in Les Revenants is missing in Ressurection. And why did they have to center it all around an FBI agent? Not enough cop shows? It may be too early to tell if the show is going to get better or not, so I will keep the watch.
Star Crossed - Oooh, another sci-fi show! This one is about an alien race that crashlands on Earth and we decide to keep them all in a concentration camp while haters roam free and try to kill them. Then an integration program is born, where Atrian teenagers are accepted into a human high school. Then it turns into a sort of Twilight meets Defiance via the Black liberation movement. It's not horrible, but it is certainly bad.
The 100 - Yet another sci-fi show about adolescents. But this one started nice enough. People live in a giant space station because they nuked the Earth. The series starts with 100 teenager death row convicts sent to the planet to ascertain if it is survivable. Lord of the Flies meets Elysium, maybe? The first episodes are intriguing, even if they already showed crass leaps in logic. I hope this one will be good, although I remember hoping the same about Lost and look how it turned out.
The After - And another sci-fi. Or so I thought. Something bad happens and some people get trapped in a building parking. They manage to escape and reach the house of one of them. Then it gets freaky when they realize they are all born on the seven of March and meet a demon like creature with a tatooed body that encompasses the individual tattoos of all of the people in the group. Also the prostitute said something about the book of Revelations, so I think it is actually a bad religious apocalypse show. Let's see where it goes.
The Assets - I first thought it was an alternative to The Americans, a show about the CIA agent Aldrich Ames, who willingly became a double agent for the KGB. The show was nice, no special effects, no artificial drama, but from the beginning it didn't seem like the TV network wanted the show to begin with. After just two episodes they cancelled it. I can't say I loved it, but it was certainly better than a lot of crap that keeps sticking to the TV screens these days.
The Americans - The second season had begun. I like the show a lot. Both main characters are fantastic in their roles and make it all believable. Their annoying daughter started to suspect things and someone just killed some undercover KGB agents and no one knows who. Keri Russell does a very good job jumping from Felicity to KGB agent, while Matthew Rhys comes out very well as a chameleon capable of impersonating just about anyone.
The Musketeers - You can't have pirates and not have musketeers. Loosely based on the Dumas books, this is a show about D'Artagnan being the coolest guy that ever lived and everybody taking an interest in him. Peter Capaldi is cardinal Richelieu. Strange choice, considering he is also Doctor Who, but very befitting the role. The show is not bad, but it feels so incredibly fake. I hope it will get better soon.
The Red Road - Jason Momoa plays in this as a bad ass native American who blackmails a cop in order to maintain his drug running operation. He plays well the role, looking all angry and violent and remorseless, but the movie subject is terribly boring. It feels like the type of show that gets cancelled and I personally have decided not to watch it anymore.
True Detective - An HBO project that features Woody Harrelson and Matthew McConaughey as cops in a God forsaken region of the US where religious fanatics and child rapists (sometimes the same person) roam free. Complex characters, eight episode seasons, very good acting and an interesting story make this one of the best shows around. It's not all rosy, though. In the first season more than three quarters was suspense and exposition. You always feel something will be going on, but it takes entire episodes before something does. It will be interesting to see if they continue with other seasons and, if they do, will they retain the same actors. The story arch has pretty much ended with the last episode of the first season.
Vikings - Season 2 has begun. Political intrigue, a new wife, backstabbing and a king of Britain who doesn't run with his tail between his legs at the sight of vikings. This season certainly keeps things interesting.
Suits - The series has never been about the law, not really, but about ritual. It was fun for a while, but every ritual, no matter how bizarre or exaggerated for dramatic purposes, can maintain outside interest only for so long. And the few law related references that made it interesting to me now are almost gone, together with the cowboyish feel of the show that made it seem less serious and more fun. Perhaps because all male characters in the series got hitched and their wings properly clipped off.
Continuum - Weird new direction of the third season of the series, with collapsing timelines, people meeting themselves and "the freelancers" being a cult like do gooder agency that protects time. I don't know yet if I like it, but it feels more confusing and less fun and sci-fi.
Crisis - A bunch of rich kids, including the son of the US president, are taken hostage by a shadowy mastermind who has thought of all possible outcomes before long before the actual execution and then uses the parents to achieve his goals, including more kidnapping of kids, I guess. Then there is this ex-cop Secret Service rookie black man who is set on finding the kids. So it's a kind of Die Hard, if you think about it. However the shadowy mastermind seems to have some moral agenda, revealing the evils of CIA, while the poor investigative agents are torn between their duty, their sympathy for the kids and their parents and the incredible stupid decisions people take in the name of their kids. I know Americans have this cult of parenthood and the need to do anything for your children, but too much is too much!
Da Vinci's Demons - The Pazzi's are trying to wrestle control of Florence from the Medici's, with Rome's involvement. That leads to some deaths, rearranging of loyalties and now both Da Vinci and Reario are heading towards the Americas on different ships, using a map of the location, only 40 years before Columbus blindly went to find the continent and with the help of none other than Amerigo Vespucci, who wasn't born yet. Nor was Da Vinci's friend, Nicolo Machiavelli, for that matter. Confusing? Only if you try to align the series with history.
Turn - This is a show about the first American spy ring, during the Revolutionary War. The first episode was rather slow, but it needed to set up the story and it showed good production values. Due to the actual nationality of the first Americans, most of the actors are British or Scottish, even if it is an AMC series. British acting with American money sounds good to me.
The Crimson Field - A six episode British miniseries about nurses in the war. The main action happens in a field hospital where four new volunteer nurses came to help out. It seems pretty decent and I already eagerly await the second episode.
Silicon Valley - An American sitcom about Silicon Valley, created by Mike Judge. Six programmers live in the same house when one of them invents an ingenious algorithm. Being an HBO series, with only eight episodes in the first season and created by Judge, I have high expectations from it.
The title of Altered Carbon refers to flesh, human flesh in particular. Richard Morgan describes a world in which people record their experiences while they live, then get transferred to other bodies when their own die or if they need to go to another planet or if they are rich enough to want a different experience or alternate bodies just for the sake of it. The alteration of the normal relationship between consciousness and physical existence is the backdrop of the book.
The good part about this cyberpunkish story is that it is personal enough and simple enough to read in one gulp. I did that, always wondering what was going to happen next and excited enough to not get distracted by something else. The bad part is that most of the props are used only to further the story and are never explored in depth. The way people are always so easily "re-sleeved" into another body and yet almost never walk with different bodies at the same time, the way "Meths" - rich people that are practically immortal, having cloned bodies waiting in case anything happens - have all this influence, but in the end fear laws and have a conscience and human failings, just as if they didn't live for centuries, and so on. I could have gotten over that more easily if all these rules would not have been waived aside whenever the main character needed them waived.
The plot is that of a detective story set in the future. A former Envoy - soldiers trained to easily switch bodies and move from planet to planet to preserve The Protectorate - is sleeved back on Earth to investigate the suspicious body-death of a rich and influential man - a Meth. During this mission, he lives dangerously, gets people to try to kill or manipulate him, women to fall for him - quite a lot actually - and in the end solves it all. So in a way, it's an interstellar James Bond.
Some of the elements in the story are haphazardly thrown around and never explained or having any connection to the main plot, like an apparent discovery of Martians, also an interstellar civilization, long gone for reasons unknown and remembered through racial memory only by whales. It was a silly proposition and pointlessly left in the book, but for me it served to show that the writer is not perfect and, even if his first book is not perfect either, it still was a nice enough read for me to do it in one swoop. Morgan has written another two books with the same character, Takeshi Kovacs, and in the other two the Martian motif is truly explored. I may end up reading them.
In conclusion, Altered Carbon is more pulp fiction than cyberpunk, with a strong backbone of detective story with a moral and a thin body of future world, disruptive technology and exploratory writing. Even if it felt naive at times, it was a pleasant read and I don't regret wasting a Saturday finishing it.
I seem to remember that I blogged about this before, but I can't find it anymore. Probably it was just a missed intention. This is simply a warning on how T-SQL converts FLOAT values to string. Here are some Transact SQL queries and their results:
SELECT @aFloat,@aDecimal,@aNumeric,@aString -- result: 1234.123456789 1234.123456789 1234.123456789 1234.123456789 SELECTCAST(@aFloat as NVARCHAR(20)),CAST(@aDecimal as NVARCHAR(20)),CAST(@aNumeric as NVARCHAR(20)),CAST(@aString as NVARCHAR(20)) -- result: 1234.12 1234.123456789 1234.123456789 1234.123456789
Wait! What happened there? The FLOAT was the only numeric format that lost precision to only 2 decimals (it is actually a loss of scale, 12345.123456789 would be converted to 12345.1). The solution is either to either convert to DECIMAL or NUMERIC values before converting to NVARCHAR or to use the STR function, which receives the scale and precision parameters. Like this:
SELECTCAST(@aFloat as NVARCHAR(20)), CAST(CAST(@aFloat asDECIMAL(18,9)) as NVARCHAR(20)), STR(@aFloat,18,9) -- result: 1234.12 1234.123456789 1234.123456789
The first conversion to DECIMAL and the STR function later on are equivalent.
I have looked into SQL options to somehow set the default precision that is used when converting a float to string, but I could not find anything usefule. Neither did settings like
SET ARITHIGNORE OFF SET NUMERIC_ROUNDABORT ON SET ARITHABORT ON
have any effect on the queries above. No error and the same result.
You don't want to know what happens with 123456789.123456789!
SELECT @aFloat, CAST(@aFloat as NVARCHAR(20)), CAST(CAST(@aFloat asDECIMAL(30,10)) as NVARCHAR(30)), STR(@aFloat,30,10) -- result: 123456789.123457 1.23457e+008 123456789.1234567900 123456789.1234567900
Not only the digits are cut even when selecting the actual value!!, but the scientific notation rears its ugly head. And look at the beautiful STR function returning ugly extra zeroes! Same issue appears when trying to use XML functions. The resulting XML has really ugly strings of the float values.
Bottom line: as much as I hate it, you probably should not use FLOAT when trying to display values. Ever.
The Windup Girl, acclaimed by many as a very good book, shows a Thailand that Paolo Bacigalupi declares as "a future version", but given scientific realities I would call it an alternate world Thailand. Wikipedia calls the book "biopunk", although I wouldn't quite call it that way, either, as the bio bits in the book didn't feel absolutely necessary to the story; more of an eco-thriller, perhaps. The book takes place in a nation that is fighting the encroaching ocean, in a time where global warming is rampant and sea levels have risen. Also, there is no more oil, no real use of electricity or combustion and everything revolves around genetics. Large elephant derivations are used to generate power; "kink-springs", a sort of mechanical energy battery, are powering just about everything; cats have been engineered to color shift to blend into their environment; human derivatives have been created, sterile, but beautiful and always healthy, slaves for things varying from military use to sex toys. But the most important element of this strange world is the overwhelming power of genetic companies. The same ones who created successful and copyrighted versions of food crops, they also released horrible diseases onto the world, making their products the only viable alternative and creating a depopulation incident.
In the book, Thailand is one of the last countries to resist the "calorie companies" through a combination of cultural and religious fanaticism, but also with the help of a hidden seed bank and a defecting company geneticist. The country is rife with political and economical tension and the main characters of the book are all caught up in this large game. You have the artificially created girl who was left behind by the Japanese and now is a sex toy to be abused every day for the pleasure of others, the AgriGen company man, his only purpose to get his hands on the seed bank, the Chinese refugee from Malaysia, where the brown skinned Muslims took over the country by ethnically cleansing anybody else, the different Thai factions and their agents, all playing the field amongst the "innocent" population of Bangkok.
The thing is that the book is not really about the "windup" genetically engineered girl, but about this world that Bacigalupi is describing. The girl herself has a pivotal role in all of this, but she is merely a secondary actor. I feel like the author wanted to give this impression of all the characters of the book, that they are transient, unimportant, even the human race as a whole, even when they are the driving force of the events around them. A very Asian perspective from a European, I guess. The writing style is good and fluent and I rarely got bored, even when the events described were not terribly exciting. The plot focuses almost exclusively on people, with the technical or logistical aspects thrown in there as afterthought. I think this is what makes the book a good one, because any inconsistency with our own world can be easily dismissed, at least for lack of evidence.
Bottom line, The Windup Girl is a very nice book, well written by Paolo Bacigalupi to describe an alternate future version of Thailand. The fantastical elements of the book are there mostly for support of the story, which in its essence is not really science fiction. One could easily imagine the same plot in a real world country, maybe modern Thailand itself. But, if you are going to write a philosophical commentary about human society and our place in the world, why not place it in an imaginary universe, as well?
I've heard of for some time now, but never actually tried anything on the site. Today I tried a course that teaches the basics of R, a statistics programming language akin to Matlab, and I thought the site was great. I suspect it all depends on the quality of the course, but at least this one was very nice. You can see my "report card" here, although I doubt I am going to visit the site very often. However, for beginners or people who quickly want to "get" something, it is a good place to start, as it gives one a "hands on" experience, like actually coding to get results, but in a carefully explained step by step tutorial format.
I was working on a pretty nice task that involved translating the text in a page in real time. For this I created a one page function that would do magic on elements that were added or changed in the page. On specific pages it moved with abysmal speed and I had no idea why. So I went to profile the thing and I was shocked to see that the problem did not come from my long piece of code, but from a simple encapsulation of an element in a jQuery object. I was using it only to have a nicer interface for getting the name of the element and changing an attribute. Here is the code:
var j=jQuery(elem); if (j.is('img[alt]')) { j.attr('alt',translate(j.attr('alt'))); }
Replaced it with:
if (/^img$/i.test(elem.tagName)) { var alt=elem.getAttribute('alt'); if (alt) { elem.setAttribute('alt',translate(alt)); } }
And it worked very fast indeed. The element might have been body so maybe the encapsulation tries to also parse the children or something like that or perhaps the problem was fixed with later versions of the library. However, think about how many times we used this kind of code without thinking twice about it. Think twice about it! :)