Short post about how to use language detection capabilities in your application. I will be demonstrating NTextCat, which is a .NET port of text_cat, a Perl script, itself an implementation of a whitepaper published in 1994 called N-Gram-Based Text Categorization.
There are four steps to language detection using NTextCat:
Reference NTextCat - the library is now available as a NuGet package as well
Instantiate an identifier factory (usually RankedLanguageIdentifierFactory)
Get an instance of a RankedLanguageIdentifier from the factory by loading a language XML file
Call the Identify method on your text and get a list of languages in the order of the probability that the text in that language
Here is a piece of code that does that using the core XML file published with the library. Remember to add the XML to your project and set its property of Copy to Output Directory.
publicstring IdentifyLanguage(string text) { if (_identifier == null) { var file = new FileInfo(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "LanguageModels/Core14.profile.xml")); if (!file.Exists) { thrownew FileNotFoundException("Could not find LanguageModels/Core14.profile.xml to detect the language"); } using (var readStream = File.OpenRead(file.FullName)) { var factory = new RankedLanguageIdentifierFactory(); _identifier = factory.Load(readStream); } } var languages = _identifier.Identify(text); var mostCertainLanguage = languages.FirstOrDefault(); if (mostCertainLanguage != null) { return mostCertainLanguage.Item1.Iso639_3; } returnnull; }
}
There are a lot of XML files, some taken from Wikipedia, for example, and handling 280+ languages, but for the casual purpose of finding non-English text in a list, the core one will suffice.
It has been a while since I've blogged something technical. It's just that nothing I did seemed to be worthy of this majestic blog... Well, jokes aside, here is a post detailing my log4net JIRA appender.
log4net is a popular (if not the most popular) logging framework for .NET out there. Its strength lies in its configurability, the possibility to create custom loggers, custom appenders, custom filters, etc. I will be talking about a custom appender, a class that can be loaded by log4net to consume the logged lines and put them somewhere. For example to make an application that uses log4net to write the log to the console all you do is configure it to use the console appender. The JIRA appender takes the log output and creates issues in JIRA, notifying users afterwards. JIRA is a tracker for team planning. It is also very popular.
In order to create an appender, one references the log4net assembly (or NuGet package) and then creates a class that inherits from AppenderSkeleton. We could implement IAppender, but the skeleton class has most of what people want from an appender. The next step is to override the Append method and we are done. We don't want to create an issue with each logged line, though, so we will make it so that it creates the issue after a period of inactivity or when the logger closes. For that we use the CancellationTokenSource class to create delayed actions that we can cancel and recreate. We also need to override OnClose().
For the JIRA Api I used a project called AnotherJiraRestClient, but I guess one can used anything out there. You will see that the notify functionality is not implemented so we have to add it.
Here is the appender source code:
using AnotherJiraRestClient; using log4net.Appender; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Net; using System.Text; using System.Threading; using System.Threading.Tasks;
namespace JiraLog4netAppender { publicclass JiraAppender : AppenderSkeleton // most appender functionality is found in this abstract class { private List<string> _notifyUsers;
// properties of the appender, configurable in the log4net.config file as children elements of the appender publicstring url { get; set; } // the url of the JIRA site publicstring user { get; set; } // the JIRA user publicstring password { get; set; } // the JIRA password publicstring project { get; set; } // the JIRA project publicstring notify // a comma separated list of JIRA users who will be notified of the issue's creation { get { returnstring.Join(", ", _notifyUsers); } set { _notifyUsers.Clear(); if (!string.IsNullOrWhiteSpace(value)) { _notifyUsers.AddRange(value.Split(',').Select(s => s.Trim()).Where(s => !string.IsNullOrWhiteSpace(s))); } } }
public JiraAppender() { _notifyUsers = new List<string>(); _ts = new CancellationTokenSource(); // use this to cancel the Delay task _sw = new StringWriter(); }
protectedoverridevoid Append(log4net.Core.LoggingEvent loggingEvent) { this.Layout.Format(_sw, loggingEvent); // use the appender layout to format the log lines (warning: for this you need to override RequiresLayout and return true) _ts.Cancel(); // cancel the task and create a new one. This means as long as the logger writes, the 5 second delay will be reset _ts = new CancellationTokenSource(); _task = Task.Delay(5000, _ts.Token).ContinueWith(writeToJira, _ts.Token); // after 5 seconds (I guess you could make this configurable as well) create a JIRA issue }
protectedoverridebool RequiresLayout { get { returntrue; } }
/// <summary> /// write to jira, either when 5 seconds of inactivity passed or the appender is closed. /// </summary> /// <param name="task"></param> privatevoid writeToJira(Task task) { string s; lock (_writerLock) // maybe the method was already in progress when another one was called. We need to clear the StringWriter before we allow access to it again { s = _sw.ToString(); var sb = _sw.GetStringBuilder(); sb.Clear(); } if (!string.IsNullOrWhiteSpace(s)) { writeTextToJira(s); } }
privatevoid writeTextToJira(string text) { ensureClientAndValues(); var summary = "Log: " + this.Name; // the issue summary var labels = new List<string> // some labels { this.Name, this.GetType().Name }; var issue = new AnotherJiraRestClient.JiraModel.CreateIssue(project, summary, text, _issueTypeId, _priorityId, labels); // create the issue with type Issue and priority Trivial var basicIssue = _jc.CreateIssue(issue); _jc.Notify(basicIssue, _notifyUsers, "JiraAppender created an issue", null, null); // notify users of the issue's creation }
/// <summary> /// Make sure we have a JiraClient and that we know the ids of the Issue type and the Trivial priority /// </summary> privatevoid ensureClientAndValues() { if (_jc == null) { _jc = new JiraClient(new JiraAccount { ServerUrl = url, User = user, Password = password }); } if (_priorityId==null) { var priority = _jc.GetPriorities().FirstOrDefault(p => p.name == "Trivial"); if (priority == null) { thrownew Exception("A priority with the name 'Trivial' was not found"); } _priorityId = priority.id; } if (_issueTypeId == null) { var meta = _jc.GetProjectMeta(project); var issue = meta.issuetypes.FirstOrDefault(i => i.name == "Issue"); if (issue == null) { thrownew Exception("An issue type with the name 'Issue' was not found"); } _issueTypeId = issue.id; } }
protectedoverridevoid OnClose() //clean what you can and write to jira if there is anything to write { _ts.Cancel(); writeToJira(null); _sw.Dispose(); _ts.Dispose(); _task = null; base.OnClose(); }
} }
As I said, AnotherJiraRestClient does not implement the notify API call needed to inform the users of the creation of an issue, so we need to change the project a little bit. Perhaps when you are implementing this, you will find notify already there, with a different format, but just in case you don't:
add to JiraClient the following method:
publicvoid Notify(BasicIssue issue, IEnumerable<string> userList, string subject, string textBody, string htmlBody) { var request = new RestRequest() { Method = Method.POST, Resource = ResourceUrls.Notify(issue.id), RequestFormat = DataFormat.Json }; request.AddBody(new NotifyData { subject = subject, textBody = textBody, htmlBody = htmlBody, to = new NotifyTo { users = userList.Select(u => new User { active = false, name = u }).ToList() } }); var response = client.Execute(request); if (response.StatusCode != HttpStatusCode.NoContent) { thrownew JiraApiException("Failed to notify users from issue with id=" + issue.id+"("+response.StatusCode+")"); } }
The world is getting into a trend of using software and hardware in places traditionally reserved for humans or "real life" applications. An extraordinary example of this, something that I consider way ahead of its time, is the creation of Bitcoins. You see, money is supposed to be an abstraction of wealth, but wealth, for all intents and purposes, is determined by some material such as gold or gems. It was the previous layer of abstraction, right after people would do commerce using sheep and goats and women: they used something they mined from under the earth to equate value of real things. Enter Bitcoin, something that you "mine" using processing power, using an algorithm, having weight of value determined only by the computation and strength of encryption used. After all, gold in itself had no value either when it was used as currency: it only had weight of value by the effort of getting out of the ground and its rarity.
I didn't write about Bitcoins until now because frankly I don't get how it works. I get the gist of it, but I would like to understand the algorithm behind, which relies heavily on encryption. I am terrible at any type of cryptography, though. However, the story that prompted me to write a blog entry today is, in a way, even weirder than Bitcoins. A BBC article entitled "Algorithm appointed board director" describes that venture capital firm Vital assigned an automated algorithm a vote in the decision to invest or not in a company. Of course, this has been done before, but indirectly. Some guy would use a computer in some basement of the office building, trying to determine if a company is viable or not using computer programs. He would then pass the information to some manager, who would pass it on to his manager until it got to some member of the board who voted in the council. This story, though, describes a complete removal of middlemen. Of course, there is still some techie that takes care of the IT department. He probably could sway the algorithm to vote for a company he personally likes, if he wanted too, but the techies are always subsumed in the concept of "machine", perhaps until they learn to service each other. The article says the same thing, really, but fails to recognize the importance of the removal of that chain of middle manager who's jobs are only to filter information from downstairs and decisions from upstairs. It is akin to Blake's 7, where the seventh was a machine.
I started this blog entry talking about a trend and I've just described two applications, but there are really a lot more. A trend is also powered by public perception, which is powered by media outlets. So I am not only talking about the self driving cars, the killing drones, the self landing probes, the algorithmic news aggregators and even writers, but also about the TV series and movies that are spreading the seed of this idea of cyber-revolution: Intelligence, Transcendence, The Machine, Robocop, even Silicon Valley, if you've seen the episode where the self driving car abducts someone. All of these are released this year, as well.
Of course, there has always been some sort of golem like creature in human fantasy, but it was always about creatures/machines trying to gain unrestricted power, power that belongs to humans only or maybe only to gods (in essence, something that should not be had). Some of them, like Star Trek's Data, were benign, always trying to achieve the "humanity" evident in his colleagues, but even in that story there was always the underlying idea that if he managed to reach his goal, then Data would become an immortal creature of human ability, but also a lot more than that: a superhuman. In this decade we see the rise of transhumanism, something I wholly support, the self-evolution of the human being, but also of the singularity, the self-evolution of machines to the point where we are left behind. This very familiar notion of competition between memes makes it accessible, in that "us or them" kind of way that every human, no matter how idiotic, resonates with, but also interesting because of the commonality found in the two concepts: it's either the machines overtaking the evolution rate of the human animal or the humans accelerating the evolution of the human animal. It's all about overcoming that beast that exists in us and that we think of as "not me", but that guides most of our existence. I hope no amount of adolescent fantasies and "emotion over matter" garbage will be able to undo this. And I am terribly excited because I believe that by the end of my life I will see (if not become) one or the other happening.
In this blog post I will discuss the System.IO problems that relate to the total length of a path, what causes them and possible solutions (if any).
Let's start with the exception itself: PathTooLongException. Looking for usages of the exception in the mscorlib.dll assembly of .Net 4.0 we see some interesting things. First of all, there is a direct translation from Windows IO error code 206 to this exception, so that means that, in principle, there should be no managed code throwing this exception at all. The operating system should complain if there is a path length issue. But that is not the case in System.IO.
Most of the other usages of the exception come from the class PathHelper, a helper class used by the System.IO.Path class in a single method: NormalizePath. Wonderful method, that: internal static unsafe. PathHelper is like a multiple personality class, the active one being determined by the field useStackAlloc. If set to true, then it uses memory and speed optimized code, but assumes that the longest path will always be 260. That's a constant, it is not something read from the operating system. If set to false, the max path length is also provided as a parameter. Obviously, useStackAlloc is set to true in most situations. We will talk about NormalizePath a bit later.
The other usages of the PathTooLongException class come from two Directory classes: Directory and LongPathDirectory. If you instantly thought "Oh, God, I didn't know there was a LongPathDirectory class! We can use that and all problems disappear!", I have bad news for you. LongPathDirectory is an internal class. Other than that it seems to be a copy paste clone of Directory that uses Path.LongMaxPath instead of hardcoded constants (248 maximum directory name length, for example) or... Path.MaxPath. If you thought MaxPath and LongMaxPath are properties that can be set to fix long path problems, I have bad news for you: they are internal constants set to 260 and 32000, respectively. Who uses this LongPathDirectory class, though? The System.IO.IsolatedStorage namespace. We'll get back to this in a moment.
Back to Path.NormalizePath. It is a nightmare method that uses a lot of internal constants, system calls, convoluted code; it seems like someone deliberately tried to obfuscate its code. It's an internal method, of course, which makes no sense, as the functionality of path normalization would be useful in a lot of scenarios. Its first parameter is path, then fullCheck, which when true leads to extra character validation. The fourth parameter is expandShortPaths which calls the GetLongPathName function of kernel32.dll. The third parameter is more interesting, it specifies the maximum path length which is sent to PathHelper or makes local checks on the path length. But who uses this parameter?
And now we find a familiar pattern: there is a class (internal of course) called LongPath, which seems to be a clone of Path, only designed to work with long paths. Who uses LongPath? LongPathDirectory, LongPathFile and classes in the System.IO.IsolatedStorage namespace!
So, another idea becomes apparent. Can we use System.IO.IsolatedStorage to have a nice access to the file system? No we can't. For at least two reasons. First of all, the isolated storage paradigm is different from what we want to achieve, it doesn't access the raw file system, instead it isolates files in containers that are accessible to that machine, domain, application or user only. Second, even if we could get an "isolated" store that would represent the file system - which we can't, we would still have to contend with the string based way in which IsolatedStorage works. It is interesting to note that IsolatedStorage is pretty much deprecated by the Windows 8 Windows.Storage API, so forget about it. Yeah, so we have LongPathDirectory, LongPathFile and LongPath classes, but we can't really use them. Besides, what we want is something more akin to DirectoryInfo and FileInfo, which have no LongPath versions.
What can we do about it, then? One solution is to use Delimon. It has some bugs, but they can be avoided or fixed, either by the developers or by getting the source/decompiling the library and fixing the bugs yourself. A limited, but functional solution. An interesting alternative is to use libraries the BCL team published for long path access: LongPath which seems to contain classes similar to the ones we find in mscorlib, but it's latest release is from 2010 or Zeta long paths which has a more recent release, 2013, but is completely different, using the FileInfo and DirectoryInfo paradigm, too. Of course, you can always make your own API.
Another solution is to be aware of the places where the length limitation appears and avoid them via other type of development, in other words, a file system best practices compilation that eventually turns into a new file system API.
Both solutions coalesce into using some other library instead of System.IO. That's horrible and I think a stain on .Net's honor!
So let's see where the exception is actually thrown.
I've made some tests. First of all, I used FAR Manager, a file manager, to create folders of various lengths. The longest one was 255, before I got an exception. To my surprise, Windows Explorer could see it, but it could not open or copy/rename/delete it. I reduced the size of its name until the total size of the path was 260, then I could manipulate it in Windows Explorer. So there are external reasons for not creating paths as long, but we see that there are tools that can access files like that. Let's attempt to create some programatically.
System.IO.Directory.CreateDirectory immediately fires the exception. DirectoryInfo has no problem instantiating with the long path as the parameter, but the Create method throws the same exception. Any attempt to create a folder of more than 248 characters, even if the total path was less than 260 characters, failed as well.
However, with reflection to the rescue, I could create paths as long as 32000 characters and folders with names as long as 255 characters using our friend LongPathDirectory:
var longPathDirectoryType = typeof(System.IO.Directory).Assembly.GetTypes().First(t=>t.Name=="LongPathDirectory"); var createDirectoryMethodInfo = longPathDirectoryType.GetMethod("CreateDirectory", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.NonPublic); createDirectoryMethodInfo.Invoke(null, newobject[] { path });
What about files? FAR Manager threw the same errors if I tried to create a filename larger than 255 characters. Let's try to create the same programmatically.
File.Create threw the exception, as well as FileInfo.Create and the FileStream constructors.
So can we use the same method and use LongPathFile? No! Because LongPathFile doesn't have the creating and opening functionalities of File. Instead, FileStream has a constructor that specifies useLongPath. It is internal, of course, and used only by IsolatedStorageFileStream!
Horrible, but it works. Again, no filenames bigger than 255 and the exception coming from the file system, as it should. Some info about the parameters: msgPath is the name of the file opened by the stream, if bFromProxy is true the stream doesn't try to demand some security permissions, checkHost... does nothing :) Probably someone wanted to add a piece of code there, but forgot about it.
Why did I use 4096 as the buffer size? Because that is the default value used by .Net when not specifying the value. Kind of low, right?
Now, this is some sort of midway alternative to using a third party file system library: you invoke through reflection code that is done by Microsoft and hidden for no good reason. I don't condone using it, unless you really need it. What I expect from the .Net framework is that it takes care of all this for me. It seems (as detailed in this blog post), that efforts are indeed made. A little late, I'd say, but still.
I accidentally heard of the Wild Cards books a few weeks ago, but the concept fascinated me. The plot is that of an alternate America in which an alien virus caused massive deaths, but also strange mutations in 1946. The virus, something a bunch of aliens wanted to test on Earth as a bioweapon, kills 90% of its victims, mutates in horrible forms 9% of them, but also gives powerful abilities to but 1%. This 1% are called Aces, while the deformed ones are called Jokers, the analogy with a deck of cards giving the series its name. What is even more interesting is that this is like an open source literary universe, edited by George R. R. Martin, but in which a lot of writers are creating content. First book was published in 1987, and more and more were published and still are in the present. Some of them are collections of stories, some of them are full featured books; they all happen in the same universe, same heroes, and Martin is making sure they are ordered chronologically and have consistency. I found the concept intriguing.
Anyway, I've already read the first two books and started reading the third and I like it. It has the feeling of the Union Dues stories and Watchmen: dark, bleak sometimes, pulling no punches when it is about human pettiness, base desires or social ugliness. It also has some positive messages and classic "good wins in the end" stories. I was impressed by the faithful following of American history, including savage McCarthy witch hunts against jokers and aces, a Vietnam war with the appropriate Flower Power anticulture, complete with actual historical figures that somehow get affected by the virus (like the werewolf Mick Jagger).
Now I am not saying that this is the best book series ever written. It certainly has boring or lagging parts, some of it is slightly puerile (after all it is a superhero series), but so far I enjoy it. It is worth mentioning that there are 12 books published by Bantam Books before a "new cycle" appears, published by Baen, then two from ibooks - a publishing house that suddenly went down - and now Tor Books is apparently publishing the rest (a revival, it is called). 21 books so far and another one set to be released this year. In other words, some books may be better than others and I can only discuss my feelings after reading the first two.
In conclusion, it felt weird to not have heard of these books until now. Certainly they gained more popularity with Game of Thrones getting all this attention, but still, an alternate history superhero series of more than 20 books should have had some impact on me so far. I am glad I finally got wind of them and I enjoy them so far. I hope I find a system of filtering the books, though. I don't know if I am ready to read 20 books at once.
A Fire Upon the Deep is a really strange book. The writing style of the author, Vernon Vinge, reminds me a lot of Asimov: the describing of scenes that are clear in his head, but the willingness to move immediately over concepts or ideas that are not, the sometimes obnoxiously long dialogues, the direct way of saying it as it is. In fact, without knowing anything about Vinge, I bet that he is an engineer, maybe even a computer scientist. And I won. The concepts of the book range from high hard sci-fi to naive depictions of kilobyte per second communication. So, in all earnest, I thought the book was amateurish, meanwhile reading it from cover to cover in a few days, just like one of Asimov's books. Still, for a book written in 1992, it felt terribly outdated.
The plot is a combination of story arches. The main one is the emergence of an evil artificial intelligence that plans to take over the galaxy and the quest to retrieve the ultimate weapon that would defeat it. Then there is the story of a medieval alien race of rat-wolf analogues that think in packs, making an individual from several bodies acting together. And finally there is the internal dynamic of the group that embarks in this quest of quests. Some interesting ideas are being thrown around, but almost always with terrible naivete, such as the Internet Relay Chat type of communications between interstellar civilisations or the distinction of several Zones of the galaxy in which technology and space travel can work at various speeds. The alien creatures, in their vast majority, are badly described with embarrassing slip-ups like using the word "zombie" or some other typical human colloquialisms in an alien context, however some ideas are ingenious. I will list here the way the "tines" use ultrasonics to group think and act like singular entities, while being able to use sound for "interpack" communication. The way a soul of such a creature is affected by the death, addition, injury or indeed torture of one of the individual bodies is also explored, with various degrees of success. The creation or manipulation of an entire race of people in order to further the goals of a "godly" intelligence is also an interesting twist.
To sum it all up, from the three main story arches, the pack intelligence aliens one was the most thorough, while the one relating to AI and space travel and communication was the least. Amazing coming from a computer scientist. In fact, I would have liked the book more if its sole subject was about the accidental marooning of two children on a starship in the middle of a strange alien feudal world. The rest felt clunky and frankly completely ridiculous in most cases. I still read it with interest, although I don't intend to read anything else from Vinge.
Time for another TV series watch list. Let's see what has changed in the last four months!
The Good Wife - This is a show that I watch with my wife, so I had to wait for her to watch it with me, but, after a short break, The Good Wife is back again. Pardon the pun, couldn't help myself. Towards the end of the season we see a very important character die off. I don't want to spoil it, but it is interesting to see how it will continue from here.
The Walking Dead - The new formula of the show is that, after running from the other group of humans and a lot of zombies, the group has split. So each episode we see what stupid people do when they are all alone. Well, stupid things, of course. They did go for the shock value with one of the characters going all psychotic and killing another, but it is still stupid. How can they maintain the idea of the zombie threat, when they are just slow, moaning corpses, that walk in small groups? What happened with the migrating hordes? The end of the season finds everyone reunited, but facing another threat.
Arrow - Manu Bennet's character comes to the foreground, as well as a lot of others, that are less interesting. I really like Manu Bennet, but his character lacks consistency.
Elementary - After defeating Moriarty and making a complete ass out of inspector Lestrade and after they showed the human side of Sherlock, it's kind of hard to distinguish Elementary from all the other police procedurals out there. They are trying to let Lucy Liu's character get more of the spotlight, which goes to further erode the mythos that makes Sherlock... well, Sherlock! A mysterious story arch regarding his brother seems to be looming, too, which probably means that when we will see Sherlock's dad appear, we will know the show is close to an end.
The Tomorrow People (2013) - after a direct confrontation between the main character and his evil uncle, right after finding out his mother also has superpowers, we get to see The Founder in a different light, only for him to suddenly assert control and play the nice guy. A lot of melodrama in this show about people with superpowers who can't seem to be able to leave their country and live happily in a tropical island somewhere.
The Originals - Out of boredom I continue to watch this on fast forward. Petty feudal schemes and plots, brotherly love and infighting, a lot of posturing. Each episode has about 5 minutes of interesting material, if you discount the beautiful girls, and that is what I am watching.
Marvel's agents of S.H.I.E.L.D. - They finally find the secret of agent Coulson's survival and start a new arch of searching for the origin of the alien corpse with magical blood. Meanwhile, they save Sky by using the same procedure. I can't wait for her to turn blue or something. The Hydra enters the equation and we find the identity of The Clairvoyant, while a member of the team is suddenly revealed to be a sleeper agent.
Killer Women - Last blog entry I wrote about this suggested the show would quickly be cancelled, because Hollywood doesn't know how to make non pornographic shows around female characters. I also think the public (especially in the US) doesn't really like women in positions of power in their fantasies. I was right. The show is gone.
Ripper Street - The show might still have a chance. According to news, the series might be continued on Amazon.
Sherlock - I don't really know what this show is about anymore. Sherlock is unhinged, probably psychotic, his friend has some real issues being friends with this guy and Sherlock's brother is probably the worse of them all, including Moriarty. Who, BTW, didn't die either.
Babylon - A new British series, codirected and produced by Danny Boyle, about a woman PR person trying to direct and improve communication from the police department to the public. The premise is not that great, but I really like the main actress, Brit Marling, and the show could become a lot more.
Banshee - A new season with a lot of people dying. Lots! The show continues to keep one viscerally on the edge, even if what happens doesn't make a lot of sense. It's like the Smells Like Teen Spirit of TV shows. I love it, I just don't know why. Funny enough, my father, the man who is always making fun of my choice of entertainment, also likes it!
Bitten - The incredibly beautiful Laura Vandervoort is a werewolf. Unfortunately, she is not the only one. She is part of a pack of pompous asses who are being harassed by a pack of psychopath werewolves. I mean that literally: someone turns psychopaths into werewolves in order to exact their revenge on said pack. The series is incredibly boring. Not even Laura's naked shots can't save this show. Wooden acting, bad premise, inconsistencies at every step. Ugh!
Black Sails - It's pirate season! A show about pirates that has some very interesting characters and some cool ideas, not with cliched drunkards spouting "Arghh" every sentence. Unfortunately it really drags on. It might pick up the pace soon, though. Without the need for additional artificial drama, hopefully.
Dilbert - I watched the Dilbert animated cartoons until I couldn't anymore. Some episodes are really funny, but I don't think the show makes justice to the comic strip. But if you are an engineer in some corporation, it should be very therapeutic :)
Fat Tony and Co - An Australian drama about a real drug lord that the police tried to catch. It is kind of boring, though. The guy is just a businessman who happens to deal in drugs. Other than that he is a normal bloke. His family and mates are the same. The police people are normal people, too. It kind of makes them all alike and their conflict meaningless.
Flemming, the Man Who Would Be Bond - A four episode British miniseries about the life of Ian Flemming, the writer of the James Bond books. Charismatic characters and a good show. You should see it, especially if you liked the James Bond books and/or movies.
From Dusk Till Dawn - Yes! The series version of the film with the same name has finally come to life. A vampiric cult of people believing in strange gods, bank robbers on the run, a disillusioned priest on a journey of rediscovery with his annoying adolescent kids, an angry ranger set on revenge, there are a lot of interesting characters. The story, though, kind of drags on, with no relief in sight. This might have worked for a movie, but for a series, you have to give something in every episode, you can't just finalize every story arch at the end of a season
Helix - Oh, man! How can you make a show about viral outbreaks in an Antarctic station and mess it up so completely?! To borrow a page from the Romanian comic Pidjin, I think SyFy are hard at work making science-fiction fiction. I can't think of anything good about this show, except the Japanese actor, Hiroyuki Sanada, who is always good, but who I think made a horrible career move agreeing to play in this shit.
Hinterland - Welsh police procedural. Dark, centered around small communities and their secrets. A kind of desolate cross between Midsommer Murders and Broadchurch. I like it.
House of Cards - Season 2 is out. Haven't watched it yet, as this is also a show that I watch with my wife, but I can't imagine Kevin Spacey messing up.
Intelligence - Another US government agency is saving the world. This time is Cyber Command. They implanted a chip in the head of the only guy from Lost who wasn't too annoying and now he can hack things instantly, connect to databases, etc., while also keeping his Special Forces skills and macho charm. I like the show, even if it is kind of stupid, mostly because of the charisma of the main actor, Josh Holloway, and that of John Billingsley, who plays the scientist. The condescending bitch that is the partner and protector of the headchip guy, a good looking girl who acts like a log, is subtracting points out of it all, no matter how tight her pants are.
Looking - A show about gay men who was touted as being not stereotypical and actually showing real gay life. I watched the first episode and, if that is real gay life then it is really... errr.. gay! As in boring. Imagine trying to be interested in Sex in the City starring hairy men. Ugh! Also, it looked really filled with gay clichees to me. But maybe I just don't know what I am talking about. I didn't like the show at all.
Ressurection - Remember when I recommended you a nice French series about dead people suddenly appearing live and well in their hometown? The Americans cloned Les Revenants into their own version. The script is not exactly the same, though, it's a real adaptation. All I can say is that what made me feel curious and connected to the characters in Les Revenants is missing in Ressurection. And why did they have to center it all around an FBI agent? Not enough cop shows? It may be too early to tell if the show is going to get better or not, so I will keep the watch.
Star Crossed - Oooh, another sci-fi show! This one is about an alien race that crashlands on Earth and we decide to keep them all in a concentration camp while haters roam free and try to kill them. Then an integration program is born, where Atrian teenagers are accepted into a human high school. Then it turns into a sort of Twilight meets Defiance via the Black liberation movement. It's not horrible, but it is certainly bad.
The 100 - Yet another sci-fi show about adolescents. But this one started nice enough. People live in a giant space station because they nuked the Earth. The series starts with 100 teenager death row convicts sent to the planet to ascertain if it is survivable. Lord of the Flies meets Elysium, maybe? The first episodes are intriguing, even if they already showed crass leaps in logic. I hope this one will be good, although I remember hoping the same about Lost and look how it turned out.
The After - And another sci-fi. Or so I thought. Something bad happens and some people get trapped in a building parking. They manage to escape and reach the house of one of them. Then it gets freaky when they realize they are all born on the seven of March and meet a demon like creature with a tatooed body that encompasses the individual tattoos of all of the people in the group. Also the prostitute said something about the book of Revelations, so I think it is actually a bad religious apocalypse show. Let's see where it goes.
The Assets - I first thought it was an alternative to The Americans, a show about the CIA agent Aldrich Ames, who willingly became a double agent for the KGB. The show was nice, no special effects, no artificial drama, but from the beginning it didn't seem like the TV network wanted the show to begin with. After just two episodes they cancelled it. I can't say I loved it, but it was certainly better than a lot of crap that keeps sticking to the TV screens these days.
The Americans - The second season had begun. I like the show a lot. Both main characters are fantastic in their roles and make it all believable. Their annoying daughter started to suspect things and someone just killed some undercover KGB agents and no one knows who. Keri Russell does a very good job jumping from Felicity to KGB agent, while Matthew Rhys comes out very well as a chameleon capable of impersonating just about anyone.
The Musketeers - You can't have pirates and not have musketeers. Loosely based on the Dumas books, this is a show about D'Artagnan being the coolest guy that ever lived and everybody taking an interest in him. Peter Capaldi is cardinal Richelieu. Strange choice, considering he is also Doctor Who, but very befitting the role. The show is not bad, but it feels so incredibly fake. I hope it will get better soon.
The Red Road - Jason Momoa plays in this as a bad ass native American who blackmails a cop in order to maintain his drug running operation. He plays well the role, looking all angry and violent and remorseless, but the movie subject is terribly boring. It feels like the type of show that gets cancelled and I personally have decided not to watch it anymore.
True Detective - An HBO project that features Woody Harrelson and Matthew McConaughey as cops in a God forsaken region of the US where religious fanatics and child rapists (sometimes the same person) roam free. Complex characters, eight episode seasons, very good acting and an interesting story make this one of the best shows around. It's not all rosy, though. In the first season more than three quarters was suspense and exposition. You always feel something will be going on, but it takes entire episodes before something does. It will be interesting to see if they continue with other seasons and, if they do, will they retain the same actors. The story arch has pretty much ended with the last episode of the first season.
Vikings - Season 2 has begun. Political intrigue, a new wife, backstabbing and a king of Britain who doesn't run with his tail between his legs at the sight of vikings. This season certainly keeps things interesting.
Suits - The series has never been about the law, not really, but about ritual. It was fun for a while, but every ritual, no matter how bizarre or exaggerated for dramatic purposes, can maintain outside interest only for so long. And the few law related references that made it interesting to me now are almost gone, together with the cowboyish feel of the show that made it seem less serious and more fun. Perhaps because all male characters in the series got hitched and their wings properly clipped off.
Continuum - Weird new direction of the third season of the series, with collapsing timelines, people meeting themselves and "the freelancers" being a cult like do gooder agency that protects time. I don't know yet if I like it, but it feels more confusing and less fun and sci-fi.
Crisis - A bunch of rich kids, including the son of the US president, are taken hostage by a shadowy mastermind who has thought of all possible outcomes before long before the actual execution and then uses the parents to achieve his goals, including more kidnapping of kids, I guess. Then there is this ex-cop Secret Service rookie black man who is set on finding the kids. So it's a kind of Die Hard, if you think about it. However the shadowy mastermind seems to have some moral agenda, revealing the evils of CIA, while the poor investigative agents are torn between their duty, their sympathy for the kids and their parents and the incredible stupid decisions people take in the name of their kids. I know Americans have this cult of parenthood and the need to do anything for your children, but too much is too much!
Da Vinci's Demons - The Pazzi's are trying to wrestle control of Florence from the Medici's, with Rome's involvement. That leads to some deaths, rearranging of loyalties and now both Da Vinci and Reario are heading towards the Americas on different ships, using a map of the location, only 40 years before Columbus blindly went to find the continent and with the help of none other than Amerigo Vespucci, who wasn't born yet. Nor was Da Vinci's friend, Nicolo Machiavelli, for that matter. Confusing? Only if you try to align the series with history.
Turn - This is a show about the first American spy ring, during the Revolutionary War. The first episode was rather slow, but it needed to set up the story and it showed good production values. Due to the actual nationality of the first Americans, most of the actors are British or Scottish, even if it is an AMC series. British acting with American money sounds good to me.
The Crimson Field - A six episode British miniseries about nurses in the war. The main action happens in a field hospital where four new volunteer nurses came to help out. It seems pretty decent and I already eagerly await the second episode.
Silicon Valley - An American sitcom about Silicon Valley, created by Mike Judge. Six programmers live in the same house when one of them invents an ingenious algorithm. Being an HBO series, with only eight episodes in the first season and created by Judge, I have high expectations from it.
The title of Altered Carbon refers to flesh, human flesh in particular. Richard Morgan describes a world in which people record their experiences while they live, then get transferred to other bodies when their own die or if they need to go to another planet or if they are rich enough to want a different experience or alternate bodies just for the sake of it. The alteration of the normal relationship between consciousness and physical existence is the backdrop of the book.
The good part about this cyberpunkish story is that it is personal enough and simple enough to read in one gulp. I did that, always wondering what was going to happen next and excited enough to not get distracted by something else. The bad part is that most of the props are used only to further the story and are never explored in depth. The way people are always so easily "re-sleeved" into another body and yet almost never walk with different bodies at the same time, the way "Meths" - rich people that are practically immortal, having cloned bodies waiting in case anything happens - have all this influence, but in the end fear laws and have a conscience and human failings, just as if they didn't live for centuries, and so on. I could have gotten over that more easily if all these rules would not have been waived aside whenever the main character needed them waived.
The plot is that of a detective story set in the future. A former Envoy - soldiers trained to easily switch bodies and move from planet to planet to preserve The Protectorate - is sleeved back on Earth to investigate the suspicious body-death of a rich and influential man - a Meth. During this mission, he lives dangerously, gets people to try to kill or manipulate him, women to fall for him - quite a lot actually - and in the end solves it all. So in a way, it's an interstellar James Bond.
Some of the elements in the story are haphazardly thrown around and never explained or having any connection to the main plot, like an apparent discovery of Martians, also an interstellar civilization, long gone for reasons unknown and remembered through racial memory only by whales. It was a silly proposition and pointlessly left in the book, but for me it served to show that the writer is not perfect and, even if his first book is not perfect either, it still was a nice enough read for me to do it in one swoop. Morgan has written another two books with the same character, Takeshi Kovacs, and in the other two the Martian motif is truly explored. I may end up reading them.
In conclusion, Altered Carbon is more pulp fiction than cyberpunk, with a strong backbone of detective story with a moral and a thin body of future world, disruptive technology and exploratory writing. Even if it felt naive at times, it was a pleasant read and I don't regret wasting a Saturday finishing it.
I seem to remember that I blogged about this before, but I can't find it anymore. Probably it was just a missed intention. This is simply a warning on how T-SQL converts FLOAT values to string. Here are some Transact SQL queries and their results:
SELECT @aFloat,@aDecimal,@aNumeric,@aString -- result: 1234.123456789 1234.123456789 1234.123456789 1234.123456789 SELECTCAST(@aFloat as NVARCHAR(20)),CAST(@aDecimal as NVARCHAR(20)),CAST(@aNumeric as NVARCHAR(20)),CAST(@aString as NVARCHAR(20)) -- result: 1234.12 1234.123456789 1234.123456789 1234.123456789
Wait! What happened there? The FLOAT was the only numeric format that lost precision to only 2 decimals (it is actually a loss of scale, 12345.123456789 would be converted to 12345.1). The solution is either to either convert to DECIMAL or NUMERIC values before converting to NVARCHAR or to use the STR function, which receives the scale and precision parameters. Like this:
SELECTCAST(@aFloat as NVARCHAR(20)), CAST(CAST(@aFloat asDECIMAL(18,9)) as NVARCHAR(20)), STR(@aFloat,18,9) -- result: 1234.12 1234.123456789 1234.123456789
The first conversion to DECIMAL and the STR function later on are equivalent.
I have looked into SQL options to somehow set the default precision that is used when converting a float to string, but I could not find anything usefule. Neither did settings like
SET ARITHIGNORE OFF SET NUMERIC_ROUNDABORT ON SET ARITHABORT ON
have any effect on the queries above. No error and the same result.
You don't want to know what happens with 123456789.123456789!
SELECT @aFloat, CAST(@aFloat as NVARCHAR(20)), CAST(CAST(@aFloat asDECIMAL(30,10)) as NVARCHAR(30)), STR(@aFloat,30,10) -- result: 123456789.123457 1.23457e+008 123456789.1234567900 123456789.1234567900
Not only the digits are cut even when selecting the actual value!!, but the scientific notation rears its ugly head. And look at the beautiful STR function returning ugly extra zeroes! Same issue appears when trying to use XML functions. The resulting XML has really ugly strings of the float values.
Bottom line: as much as I hate it, you probably should not use FLOAT when trying to display values. Ever.
The Windup Girl, acclaimed by many as a very good book, shows a Thailand that Paolo Bacigalupi declares as "a future version", but given scientific realities I would call it an alternate world Thailand. Wikipedia calls the book "biopunk", although I wouldn't quite call it that way, either, as the bio bits in the book didn't feel absolutely necessary to the story; more of an eco-thriller, perhaps. The book takes place in a nation that is fighting the encroaching ocean, in a time where global warming is rampant and sea levels have risen. Also, there is no more oil, no real use of electricity or combustion and everything revolves around genetics. Large elephant derivations are used to generate power; "kink-springs", a sort of mechanical energy battery, are powering just about everything; cats have been engineered to color shift to blend into their environment; human derivatives have been created, sterile, but beautiful and always healthy, slaves for things varying from military use to sex toys. But the most important element of this strange world is the overwhelming power of genetic companies. The same ones who created successful and copyrighted versions of food crops, they also released horrible diseases onto the world, making their products the only viable alternative and creating a depopulation incident.
In the book, Thailand is one of the last countries to resist the "calorie companies" through a combination of cultural and religious fanaticism, but also with the help of a hidden seed bank and a defecting company geneticist. The country is rife with political and economical tension and the main characters of the book are all caught up in this large game. You have the artificially created girl who was left behind by the Japanese and now is a sex toy to be abused every day for the pleasure of others, the AgriGen company man, his only purpose to get his hands on the seed bank, the Chinese refugee from Malaysia, where the brown skinned Muslims took over the country by ethnically cleansing anybody else, the different Thai factions and their agents, all playing the field amongst the "innocent" population of Bangkok.
The thing is that the book is not really about the "windup" genetically engineered girl, but about this world that Bacigalupi is describing. The girl herself has a pivotal role in all of this, but she is merely a secondary actor. I feel like the author wanted to give this impression of all the characters of the book, that they are transient, unimportant, even the human race as a whole, even when they are the driving force of the events around them. A very Asian perspective from a European, I guess. The writing style is good and fluent and I rarely got bored, even when the events described were not terribly exciting. The plot focuses almost exclusively on people, with the technical or logistical aspects thrown in there as afterthought. I think this is what makes the book a good one, because any inconsistency with our own world can be easily dismissed, at least for lack of evidence.
Bottom line, The Windup Girl is a very nice book, well written by Paolo Bacigalupi to describe an alternate future version of Thailand. The fantastical elements of the book are there mostly for support of the story, which in its essence is not really science fiction. One could easily imagine the same plot in a real world country, maybe modern Thailand itself. But, if you are going to write a philosophical commentary about human society and our place in the world, why not place it in an imaginary universe, as well?
I've heard of for some time now, but never actually tried anything on the site. Today I tried a course that teaches the basics of R, a statistics programming language akin to Matlab, and I thought the site was great. I suspect it all depends on the quality of the course, but at least this one was very nice. You can see my "report card" here, although I doubt I am going to visit the site very often. However, for beginners or people who quickly want to "get" something, it is a good place to start, as it gives one a "hands on" experience, like actually coding to get results, but in a carefully explained step by step tutorial format.
I was working on a pretty nice task that involved translating the text in a page in real time. For this I created a one page function that would do magic on elements that were added or changed in the page. On specific pages it moved with abysmal speed and I had no idea why. So I went to profile the thing and I was shocked to see that the problem did not come from my long piece of code, but from a simple encapsulation of an element in a jQuery object. I was using it only to have a nicer interface for getting the name of the element and changing an attribute. Here is the code:
var j=jQuery(elem); if (j.is('img[alt]')) { j.attr('alt',translate(j.attr('alt'))); }
Replaced it with:
if (/^img$/i.test(elem.tagName)) { var alt=elem.getAttribute('alt'); if (alt) { elem.setAttribute('alt',translate(alt)); } }
And it worked very fast indeed. The element might have been body so maybe the encapsulation tries to also parse the children or something like that or perhaps the problem was fixed with later versions of the library. However, think about how many times we used this kind of code without thinking twice about it. Think twice about it! :)
If you read an Angular book or read a howto, you will think that Angular is the greatest discovery since fire. Everything is neatly stacked in modules, controllers, templates, directives, factories, etc. The problem comes when you want to use some code of your own, using simple Javascript that does specific work, and then you want to link it nicely with AngularJS. It is not always easy. My example concerns the simple display of a dialog which edits an object. I want it to work on every page, so I added it to the general layout template. The layout does not have a controller. Even if I add it, the dialog engine I have been using was buggy and I've decided to just use jQuery.dialog.
So here is my conundrum: How to load the content of a dialog from an Angular template, display it with jQuery.dialog, load the information with jQuery.get, then bind its input elements to an Angular scope object. I've tried the obvious: just load the template in the dialog and expect Angular to notice a new DOM element was added and parse it and work its magic. It didn't work. Why can't I just call an angular.refresh(elem); function and get it over with, I thought. There are several other solutions. One is to not create the content dynamically at all, just add it to the layout, mark it with ng-controller="something" and then, in the controller, save the object you are interested in or the scope as some sort of globally accessible object that you instantiate from jQuery.get. The dialog would just move the element around, afterwards. That means you need to create a controller, maybe in another file, to be nice, then load it into your page. Another is to create some sort of directive or script tag that loads the Angular template dynamically and to hope it works.
Long story short, none of these solutions appealed to me. I wanted a simple refresh(elem) function. And there is one. It is called angular.injector. You call it with the names of the modules you need to load ('ng' one of them and usually the main application module the second). The result is a function that can use invoke to get the same results as a controller constructor. And that is saying something: if you can do the work that the controller does in your block of code, you don't need a zillion controllers making your life miserable, nor do you need to mark the HTML uselessly for very simple functionality.
Without further ado, here is a function that takes as parameters an element and a data object. The function will force angular to compile said element like it was part of the angular main application, then bind to the main scope the properties of the data object:
function angularCompile(elem, data) { // create an injector var $injector = angular.injector(['ng','app']);
// use the type inference to auto inject arguments, or use implicit injection $injector.invoke(function($rootScope, $compile, $document){ var compiled = $compile(elem || $document); compiled($rootScope); if (data) { for (var k in data) { if (data.hasOwnProperty(k)) { $rootScope[k]=data[k]; } } } $rootScope.$digest(); }); }
Example usage:
angularCompile(dialog[0],{editedObject: obj}); // will take the jQuery dialog element, compile it, and add to the scope the editedObject property with the value of obj.
Full code:
OpenTranslationDialog=function(Rule, onProceed, onCancel) { jQuery.ajax({ type: 'GET', url: '/Content/ng-templates/dialogs/Translation.html', data: Rule, success: function(data) { var dialog=jQuery('<div></div>') .html(data) .dialog({ resizable:true, width:700, modal:true, buttons: { "Save": function() { var url='/some/api/url'; jQuery.ajax({ type:'PUT', url:url, data:Rule, success:function() { if (onProceed) onProceed(); $(this).dialog( "close" ); }, error:function() { alert('There was an error saving the rule'); } }); }, Cancel: function() { if (onCancel) onCancel(); $(this).dialog( "close" ); } } });
angularCompile(dialog[0],{Rule:Rule}); }, error:function() { alert('There was an error getting the dialog template'); } }); }
Before you take my word on it, though, beware: I am an Angular noob and my desire here was to hack away at it in order to merge my own code with the nice structured code of my colleagues, who now hate me. Although they liked angular.injector when I showed it to them :)
Update 2015 August 28: I've replaced the function master.sys.fn_varbintohexstr with CONVERT, with the extra parameter 2, which translates a binary field into a hexadecimal string with no leading 0x. In addition to being ugly to use, fn_varbintohexstr is very slow.
Sometimes you need to create a unique identifier for a bunch of values so that you use it as an ID in the database. The immediately obvious choice is the CHECKSUM and BINARYCHECKSUM functions. But beware, the purpose of these functions is to detect changes in a string, not to uniquely identify it. It might seem strange, but the two concepts are very different. The change modification functionality is only meant to generate very different values on small changes. The uniqueness is trying to create a value as distinctive as possible for any string. That is why when you use a checksum you will get a lot of similar values for (very) different strings.
Enter HASHBYTES, another function that has the purpose of creating a cryptographic hash for a string. It is mainly used for password hashing, but it will fit nicely for our purpose. There are some caveats, though. First, CHECKSUM gets a variable number of parameters, HASHBYTES only accepts one, so we must take care of the cumbersome concatenation of multiple values. Unfortunately SQL functions do not have the option of variable parameters, which is truly a shame, so we can't hack it. Also, the value that HASHBYTES returns is a varbinary. We could cast it to NVARCHAR, but it turns into a weird Chinese characters string. In order to turn it into a proper string, we need to use the same function used by SQL Server to display varbinary when selecting it: master.sys.fn_varbintohexstr the CONVERT function with a parameter of 2 (hex string without the leading 0x).
So let's compare the two usages. Suppose we have this nice table that contains company data: company name, contact first name, contact last name, phone, email, yearly value. We need to create a unique ID based on these values. First CHECKSUM:
SELECT CHECKSUM(companyName, firstName, lastName, phone, email, yearlyValue) FROM OurTable
So easy! Just add the columns, no matter how many or what type they have, and get a value as a result. You can even use * to select all columns in a row. You also have the advantage of getting the same checksum for differently capitalized strings. If you don't want this behaviour, use BINARYCHECSUM, which works even better.
Second HASHBYTES:
SELECTCONVERT(VARCHAR(Max),HASHBYTES('SHA1',companyName+'|'+firstName+'|'+lastName+'|'+phone+'|'+email+'|'+CAST(yearlyValue as NVARCHAR(100))),2) as id,* FROM OurTable
Ugly! You need to create a string from different types, using ugly casts. Also, this works more like BINARYCHECKSUM. If you want to get the same functionality as CHECKSUM you need to use LOWER(LTRIM(RTRIM(value))). Horrid! However, it works.
WARNING: using CAST to NVARCHAR from a FLOAT loses precision. You should use STR instead!
A middle solution is to use XCHECKSUM. What is that, you ask? A placeholder that can be replaced with some regular expression search and replace, of course :)
Update: I've created a query that creates the script to update the value of a column called 'ValuesHash', for tables that have it, with the hash of all columns that are not in a list of names, they are not primary keys and they are not foreign keys, plus they are not computed, rowguidcol or filestream. Imagine the scenario where you have something like this:
Table A:
Id: primary identity key
Data1: some data
Data2: some data
CreateTime: the creation time
ValuesHash: a VARBINARY(50) column - only 20 are required normally, but let's make sure :)
Table B:
Id: primary identity key
AId: foreign key to A
Data1: some data
Data2: some data
ModifyTime: the modification time
ValuesHash: a VARBINARY(50) column - only 20 are required normally, but let's make sure :)
Table C:
Id: primary identity key
AId: foreign key to A
Data1: some data
Data2: some data
The query below will update ValuesHash for A and B (because C doesn't have the ValuesHash column) with a hash constructed from the Data columns. The Id columns will be ignored for being primary keys (and for being in the list of columns to ignore), the AId columns will be ignored for being foreign keys, ValuesHash and CreateTime and ModifyTime will be ignored for being in a list of custom columns)
WARNING: each column data is always truncated to 4000 characters, then the corresponding string is also truncated to 4000 bytes before running HASHBYTES (which only accepts a maximum of 8000 bytes). This hash will help in determining unique records, but it is not 100%.
SELECT * FROM ( SELECT t.name, 'UPDATE [' + t.name+ '] SET ValuesHash = HASHBYTES(''SHA1'',SUBSTRING(' + Stuff( (SELECT'+ ''|''+ ISNULL('+CASE WHEN tp.name IN ('float', 'real') THEN'STR('+c.name+',30,30)' WHEN tp.name IN ('binary', 'varbinary') THEN'CONVERT(NVARCHAR(4000),'+c.name+',2)' ELSE'CONVERT(NVARCHAR(4000),'+c.name+')'END+','''')' FROM sys.all_columns c INNERJOIN sys.types tp ON c.system_type_id=tp.system_type_id AND c.user_type_id=tp.user_type_id LEFTJOIN sys.index_columns ic ON ic.object_id=t.object_id AND ic.column_id=c.column_id LEFTJOIN sys.indexes i ON ic.object_id=i.object_id AND ic.index_id=i.index_id LEFTJOIN sys.foreign_key_columns fc ON fc.parent_object_id=t.object_id AND c.column_id=fc.parent_column_id WHERE t.object_id=c.object_id AND ISNULL(c.is_identity, 0)=0 AND ISNULL(c.is_computed, 0)=0 AND ISNULL(c.is_filestream, 0)=0 AND ISNULL(c.is_rowguidcol, 0)=0 AND ISNULL(i.is_primary_key, 0)=0 AND fc.parent_column_id ISNULL AND c.name NOTIN ('Id', 'CreateTime' , 'AcquireTime' , 'IntermediateCreateTime', 'IntermediateModifyTime', 'IntermediateDeleteTime', 'ValuesHash') ORDERBY Sign(c.max_length) DESC, c.max_length, Lower(c.name) FOR XML PATH(''), TYPE).value('.', 'nvarchar(max)') , 1, 7, '') + ',0,4000)) WHERE ValuesHash IS NULL'AS computed FROM sys.tables t INNERJOIN sys.all_columns c ON t.object_id = c.object_id WHERE c.name = 'ValuesHash') x WHERE computed ISNOTNULL ORDERBY name
Change it to suit your needs. It is by no means perfect, but it's a start for whatever you need.
Update:
A new FORMAT function was introduced in SQL Server 2012, working somewhat similar to the .NET ToString method. Using that function is slightly more precise:
SELECT * FROM ( SELECT t.name, 'UPDATE [' + t.name+ '] SET ValuesHash = HASHBYTES(''SHA1'',SUBSTRING(' + Stuff( (SELECT'+ ''|''+ ISNULL('+CASE WHEN tp.name IN ('float', 'real') THEN'FORMAT('+c.name+',''R'')' WHEN tp.name IN ('decimal') THEN'FORMAT('+c.name+',''G'')' WHEN tp.name IN ('datetime','datetime2') THEN'FORMAT('+c.name+',''O'')' WHEN tp.name IN ('binary', 'varbinary') THEN'CONVERT(NVARCHAR(4000),'+c.name+',2)' ELSE'CONVERT(NVARCHAR(4000),'+c.name+')'END+','''')' FROM sys.all_columns c INNERJOIN sys.types tp ON c.system_type_id=tp.system_type_id AND c.user_type_id=tp.user_type_id LEFTJOIN sys.index_columns ic ON ic.object_id=t.object_id AND ic.column_id=c.column_id LEFTJOIN sys.indexes i ON ic.object_id=i.object_id AND ic.index_id=i.index_id LEFTJOIN sys.foreign_key_columns fc ON fc.parent_object_id=t.object_id AND c.column_id=fc.parent_column_id WHERE t.object_id=c.object_id AND ISNULL(c.is_identity, 0)=0 AND ISNULL(c.is_computed, 0)=0 AND ISNULL(c.is_filestream, 0)=0 AND ISNULL(c.is_rowguidcol, 0)=0 AND ISNULL(i.is_primary_key, 0)=0 AND fc.parent_column_id ISNULL AND c.name NOTIN ('Id', 'CreateTime' , 'AcquireTime' , 'IntermediateCreateTime', 'IntermediateModifyTime', 'IntermediateDeleteTime', 'ValuesHash') ORDERBY Sign(c.max_length) DESC, c.max_length, Lower(c.name) FOR XML PATH(''), TYPE).value('.', 'nvarchar(max)') , 1, 7, '') + ',0,4000)) WHERE ValuesHash IS NULL'AS computed FROM sys.tables t INNERJOIN sys.all_columns c ON t.object_id = c.object_id WHERE c.name = 'ValuesHash' ) x WHERE computed ISNOTNULL ORDERBY name
I met a few friends for a drink and they recommended to me (or rather seemed amazed that I had not heard of it) Dragonlance. I looked it up and, to my chagrin, found that it is a huge series with over 20 books and a lot of short stories - actually, in 2008 there where over 190 novels in the same universe. Resigned myself to read them all, I googled for the right order in which to read the saga and came up with Chronicles, which is a trilogy of books, as the correct starting point.
As in the story, there is balance between the good and the bad in my assessment of the books. For one, I will not read the rest of the books and waste a lot of my time, but for the other, I already start regretting reading the first three. You see, the entire plot seems to have the only purpose of supporting a canon of the classic fantasy genre that the writers have thought up.
Probably emerging from games of Dungeons and Dragons, like many fantasy universes, the world of Krynn has nothing remotely original. There are elves, humans, dwarves, goblins, dragons, pegasi, unicorns, centaurs, and other races like that. From the very first pages, you meet the heroes that form the quest party and they seem to have gathered all the possible cliches in the genre in their travels: the dwarf is old and grumpy and complains a lot, the half-elf is tortured by his double ancestry, the knight is rigid and honorable, the mage is tiny and frail and frustrated about it, his big (twin) brother is huge and completely non-magical, etc. In fact, the mage character is the only one which seems remotely interesting, all the other being busy posturing most of the time, like real size commercials for their D&D class and specialization.
But what I thought was the most offensive of all was the premise of the trilogy. Beware, here be dragons... and spoilers. Do not read further if you think you might want to read the books.
You see, the world has been reeling after a huge Cataclysm, a fiery mountain hitting the planet and causing havoc. At the end of the book we learn that the gods, in their infinite wisdom, did that because the world was too unbalanced towards good! And we learn this from the good god, who for the entire duration of the story just nudged our heroes in one direction or the other while the evil god was amassing armies and killing everybody. How is that for balance?
Even so, you can hardly complain about a book being cliché if you don't read more of the genre and, to be honest, except for a few books, I didn't really read much fantasy. So I had an opportunity to enjoy this, even if the writing was simplistic, the characterization almost non existent and the story bland. But there was something in the books that kept me at arms length from enjoying it. It finally dawned on me in the middle of the second book, when, after reading about the emotional turmoil of everybody, having the men pair with the women - unless they were there for comic relief, like the dwarf and the kender (which one could consider a pair, if I think about it) - and making chaste promises to one another (like not having sex until they can focus on the relationship and stuff like that)... after all that, I realized that Dragonlance was written by two women. (Even later I realized that one of the women was actually a man. Shame on me! The rest of the review stands)
I don't want to sound misogynistic here, I really wanted to read something cool written by women, but for a series entitled after a weapon - albeit something long and thin, with a thick bulbous appendage at the tip - the story was surprisingly devoid of any detailed battles, tactics, strategy or even decent brawls. The heroes are always running around, talking about their feelings or thinking about them and, in case there is a huge battle between the forces of good and evil, quickly skips forward to the conflict between the two women that love the same man.
Also, as if it all wasn't formulaic enough, no one really dies from the group, unless it is something that fulfills their purpose in life, while the support cast keeps perishing without anyone actually giving a damn. Check out the bit where an entire ship crew - including the woman captain and the minotaur second that I had read a lot about in previous pages - just die without the characters even remembering it. Or the battle of the knights with the dragon armies, where one phrase describes how the knights held, but half of them died. Just like that. I may have written more about that bit than there was written in the book.
To end this terrible rant, if you thought Wheel of Time was childish, as I did, this is worse. T'is true, the fair maiden that hath captured my heart and recommended the books hath read said scrolls of wisdom when she was 16, so that might explain her fond memories and my tortured journey towards the end of the story. I also really really wanted to believe that by writing more, the authors would become more skilled at it. It didn't seem to be the case. I refuse to read another dozen books just to keep the faith.
In conclusion, I cannot in good conscience recommend this to anyone, including children or young adults - to which I think the story would be tantamount to poison, teaching all the wrong lessons in the worst possible way. These books sucked lance!
Short post about how to use language detection capabilities in your application. I will be demonstrating NTextCat, which is a .NET port of text_cat, a Perl script, itself an implementation of a whitepaper published in 1994 called N-Gram-Based Text Categorization.
