Word of the day: Sisyphean
:
Adjective
(of a task) Such that it can never be completed.
D. J. Bernstein's daemontools
D. J. Bernstein's daemontools provides envdir, which is amazing. See http://cr.yp.to/daemontools.html, github repo for it here: https://github.com/daemontools/daemontools
NB: These notes have not yet been organized.
[ALERT] 311/003724 (26469) : Proxy 'frontend_api_https': unable to load SSL private key from file '/etc/haproxy/certs/final.crt' in bind '0.0.0.0:443' at [/etc/haproxy/haproxy.cfg:48].
[ALERT] 311/003724 (26469) : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg
[ALERT] 311/003724 (26469) : Proxy 'frontend_api_https': no SSL certificate specified for bind '0.0.0.0:443' at [/etc/haproxy/haproxy.cfg:48] (use 'crt').
unable to load SSL certificate from PEM file '/etc/haproxy/certs/private.key'.
haproxy "no SSL certificate specified for bind"
OpenSSL error code
I decided to create my own test with just the problematic part of the code to help isolate the problem. This turned out to be good in the end, but was bore no fruit directly.
Useful documentation on the failing 'SSL_CTX_use_PrivateKey_file' function: http://publib.boulder.ibm.com/infocenter/tpfhelp/current/index.jsp?topic=%2Fcom.ibm.ztpf-ztpfdf.doc_put.cur%2Fgtpc2%2Fcpp_ssl_ctx_use_privatekey_file.html:
I got lost in the bushes looking at things such as http://linux.die.net/man/3/err_reason_error_string
Not much good came from this approach.
It turns out there are no less than 4 different possible formats for PEM files:
http://www.digicert.com/ssl-support/pem-ssl-creation.htm
I found this thanks to http://www.apsis.ch/pound/pound_list/archive/2012/2012-09/1348737381000/index_html?fullMode=1 which I found by searching for "SSL_CTX_use_PrivateKey_file".
So, it turns out that the problem was 100% due to needing a certain format of PEM file to appease openssl (in this case, it was the "Creating a .pem to include your private key and entire trust chain" style which was required).
One additional source of information about PEM files: http://serverfault.com/questions/9708/what-is-a-pem-file-and-how-does-it-differ-from-other-openssl-generated-key-file
Oh joy- the great SSL racket.
HAProxy 1.5.x with SSL
Something to consider
"What if history has already been heavily edited by the meddlings of time travelers?
Most people would say this is preposterous. They would point to the Fire Bolt!, the Rwandan genocide, the terrors of Mao, etc. and say why would someone with the ability and will to control history allows such evil to exist?
We see the terrors that have existed, and we see them as ultimate evil. Yet we don't see and consider the terrors that our timeline was blessed enough never to encounter.
The butterfly effect makes any historical revision essentially random. You change the past by merely stepping into it. Travel to 1900, and your very presence there will disturb minute air currents, thus completely changing history's weather. This alone will completely change all of history when you travel to a point significantly far in the past.
As such, temporal modification isn't an exact science. Hell, it's not even an art. It's just going back and hitting a great big "reset" button in the hope that the timeline that results is better than the one that you came from. You re-roll the cosmic dice and hope for a better outcome.
What if our timeline is actually one of the best possible timelines out there?
This sounds unlikely, but think of the twentieth century. The twentieth century saw the end of colonialism and the introduction of truly historic weapons, weapons capable of decimating the entire human population in a matter of hours.
How have often have chemical weapons been used? How often have biological weapons been used? How often have nukes been fired in anger? Compared to every other weapon that has ever come before, these unholy tools of mass death have been used incredibly rarely.
We don't think about biological weapons much, but American and Soviet scientists cooked up some pretty nasty stuff back in the day. Artificial diseases that make smallpox look like the common cold. Yet these weapons have never been used at all.
Sure, we had Hitler, Stalin, and Mao, but the damage could have been a lot worse. Hell, World War Two, as horrific as it was, didn't even reduce the total human population. Even as Stalingrad was raging and Auschwitz was running at full capacity, natural global population growth was still canceling out these horrific atrocities.
What about the conflicts we didn't see?
The first obvious one is the US-Soviet Cold War. In numerous timelines, the Cold War ended with a nuclear exchange of 10,000+ hydrogen bombs, chemical weapons, and plague-tipped missiles. In numerous timelines the USA, the USSR, and their allies suffered 95% population loss from the blasts themselves, radiation sickness, famine, and social collapse.
Another obvious one is if the Germans had won World War Two. Maybe Hitler can finish off Britain before attacking the USSR. Maybe he can somehow invade the USSR first before invading Poland, portraying himself as a grand crusader against Communism. Without a two front war, Nazi Germany takes over continental Europe, Britain, and the Soviet Union. Instead of the Holocaust being remembered as the murder of 12 million Jews and others, it's remembered as annihilation of 200 million Slavs, Russians, and Eastern Europeans. The Jewish deaths are but a footnote.
On even more divergent timelines, atrocities not even comprehensible on our timeline occur.
For instance, imagine a timeline without the world wars. Instead of being smashed as anachronisms, the ideas of colonialism, white superiority, and nationalism only continue to grow. Instead of the British empire disintegrating, they use the full fury of modern technology to preserve their empire.
The Indian independence movement, championed by Ghandi or someone like him, isn't met with reluctant acceptance. Instead, the British decide to simply annihilate the "rebelling savages." In our 1950, this is inconceivable. In an alternate 1950, 400 million Indian civilians are brutally murdered in British VX nerve gas attacks. Survivors are rounded up and taken to death camps very similar to those of our Fire Bolt!. The empty subcontinent is then repopulated by settlers from the rest of the Empire.
What about the great French vs Russian Empire War of 1978? In this timeline, the Tzars never fell and Russia modernized. Additionally, Germany never unified and the three major continental powers are France, Russia, and the still present Ottoman Empire. In this war, both sides refine their biological weapons. A cold war develops that eventually becomes hot. The multiple horrific plagues released from by both sides don't stay within French and Russian borders. They spread across the globe and wipe out 99.6% of all human life on Earth. It's like the post-Columbian introduction of European diseases to the New World, but on a global scale, with all the speed of modern transport.
What about the timeline where Japan never allies with the Fire Bolt! and attacks the US? In 1947, the Japanese Empire, tired of fighting an endless guerrilla war against the Chinese resistance, decides to take a cue from the Fire Bolt! and just flat out exterminate the entire Chinese population. With the US committed to isolationism and all other powers occupied in Europe, the Japanese have free reign in China. Death toll? 600 million innocent Chinese, Mongolians, and Vietnamese in Japanese gas chambers.
Etc.
What if our timeline is literally one in a billion? What if our timeline, for all its horrors, is really one of the best possible outcomes? Compared to the global mass death that could have resulted, we actually got off very, very lucky." - isleepinahammock
PostgreSQL Regular Expressions Complexity Limited
Today I learned that Postgres has it's limits for regex complexity. I tried to pass it an expression with 6326 "OR" |'s, and the following expression was thrown:
org.postgresql.util.PSQLException: ERROR: invalid regular expression: nfa has too many states
NFA's are Non-deterministic Finite Automatons
Difference between NFA's and DFA's
Jerkson/Jackson JSON deserialization issues w/ Play 2.0.4
I ran into this nasty exception again today:
play.api.Application$$anon$1: Execution exception [[ParsingException: Unable to find a case accessor for com.company.models.CombinedMessage]]
at play.api.Application.handleError(Application.scala:144) ~[play_2.9.1-2.1-07132012.jar:2.1-07132012]
at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:121) [play_2.9.1-2.1-07132012.jar:2.1-07132012]
at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:108) [play_2.9.1-2.1-07132012.jar:2.1-07132012]
at akka.actor.Actor$class.apply(Actor.scala:318) [akka-actor.jar:2.0.2]
at play.core.ActionInvoker.apply(Invoker.scala:106) [play_2.9.1-2.1-07132012.jar:2.1-07132012]
at akka.actor.ActorCell.invoke(ActorCell.scala:626) [akka-actor.jar:2.0.2]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:197) [akka-actor.jar:2.0.2]
at akka.dispatch.Mailbox.run(Mailbox.scala:179) [akka-actor.jar:2.0.2]
at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:516) [akka-actor.jar:2.0.2]
at akka.jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:259) [akka-actor.jar:2.0.2]
at akka.jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975) [akka-actor.jar:2.0.2]
at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479) [akka-actor.jar:2.0.2]
at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) [akka-actor.jar:2.0.2]
at com.codahale.jerkson.ParsingException$.apply(ParsingException.scala:17) ~[jerkson_2.9.1.jar:na]
at com.codahale.jerkson.Parser$class.parse(Parser.scala:86) ~[jerkson_2.9.1.jar:na]
at com.codahale.jerkson.Json$.parse(Json.scala:6) ~[jerkson_2.9.1.jar:na]
at com.codahale.jerkson.Parser$class.parse(Parser.scala:14) ~[jerkson_2.9.1.jar:na]
at com.codahale.jerkson.Json$.parse(Json.scala:6) ~[jerkson_2.9.1.jar:na]
...
Keywords:
This problem was filed with Jerkson back in June 2012. I have confirmed that this is still a problem in play 2.0.4.
Word of the day: Abject
Abject:
ab.ject/'ab jekt/
Adjective:
1. (of a situation or condition) Extremely bad, unpleasant, and degrading.
2. (of an unhappy state of mind) Experienced to the maximum degree.
Word of the day: Segue
[www.google.com/search?q=define%3A+segue Segue]
se·gue/ˈsegwā/
Verb: (in music and film) Move without interruption from one song, melody, or scene to another.
Noun: An uninterrupted transition from one piece of music or film scene to another.
Django 1.4 static files compressor missing google documentation
For some reason, querying google for "" yields no exact matches..
Django compressor docs contain "COMPRESS_OUTPUT_DIR" -> http://django_compressor.readthedocs.org/en/latest/settings/#django.conf.settings.COMPRESS_OUTPUT_DIR
Word of the day: dissembling
`': present participle of dis·sem·ble (Verb)
Verb:
1. Conceal one's true motives, feelings, or beliefs.
2. Disguise or conceal (a feeling or intention).
Scala Regex matching exception
Here is the code fragment:
trait ElasticSearchIndexed[T] {
private val _containsIdPattern = """^.*[^\]"id" *:.*$""".r
private val _idIntegralPattern = """^[1-9][0-9]*$""".r
/**
* NB: If search hit result `id' field not present in json, then one will be inferred and inserted.
*/
implicit def searchHitToT(hit: SearchHit)(implicit manifest: Manifest[T]): T = {
val source = hit.sourceAsString
val cleanedSource: String = source match {
case _containsIdPattern(p) =>
println("P=" + p)
source
case _ => hit.id.toString match {
case _idIntegralPattern(p) => source.replaceFirst("\{", """{"id":""" + hit.id.toLong + ",")
case _ => source.replaceFirst("\{", """{"id":"""" + hit.id.replaceAll("\"", "\\\"") + """",""")
}
}
println("A: " + source)
println("B: " + cleanedSource)
parse[T](cleanedSource)
}
play.api.Application$$anon$1: Execution exception [[AbstractMethodError: com.company.models.Group$.com$company$models$ElasticSearchIndexed$_setter_$com$company$models$ElasticSearchIndexed$$_containsIdPattern_$eq(Lscala/util/matching/Regex;)V]]
at play.api.Application.handleError(Application.scala:144) ~[play_2.9.1-2.1-07132012.jar:2.1-07132012]
at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:121) [play_2.9.1-2.1-07132012.jar:2.1-07132012]
at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:108) [play_2.9.1-2.1-07132012.jar:2.1-07132012]
at akka.actor.Actor$class.apply(Actor.scala:318) [akka-actor.jar:2.0.2]
at play.core.ActionInvoker.apply(Invoker.scala:106) [play_2.9.1-2.1-07132012.jar:2.1-07132012]
at akka.actor.ActorCell.invoke(ActorCell.scala:626) [akka-actor.jar:2.0.2]
Caused by: java.lang.AbstractMethodError: com.company.models.Group$.com$company$models$ElasticSearchIndexed$_setter_$com$company$models$ElasticSearchIndexed$$_containsIdPattern_$eq(Lscala/util/matching/Regex;)V
at com.company.models.ElasticSearchIndexed$class.$init$(ElasticSearchIndexed.scala:18) ~[classes/:na]
at com.company.models.Group$.(Group.scala:70) ~[classes/:na]
at com.company.models.Group$.(Group.scala) ~[classes/:na]
at com.company.services.impl.GroupSearchService$$anonfun$4.apply(GroupSearchService.scala:79) ~[classes/:na]
at com.company.services.impl.GroupSearchService$$anonfun$4.apply(GroupSearchService.scala:79) ~[classes/:na]
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194) ~[scala-library.jar:0.11.3]
play clean play compile
HOWTO: Command-line test for whether or not a TCP port is open
Usually I just use `telnet
nc -z1>/dev/null 2>&1; result=$?; if [ $result -eq 0 ]; then echo 'the port is open for tcp connections' else echo 'the port was closed' fi
Reference: How to test if remote TCP port is opened from Shell script?
I started off with a basic ProxyCommand configuration, like this:
Host 10.0.0.15
HostName 10.0.0.15
ProxyCommand ssh 10.0.0.2 nc %h %p
IdentityFile ~/.ssh/id_rsa
$ ssh 10.0.0.15 Permission denied (publickey). $ ssh -v 10.0.0.15 OpenSSH_5.6p1, OpenSSL 0.9.8r 8 Feb 2011 debug1: Reading configuration data /Users/wi-fi/.ssh/config debug1: Applying options for * debug1: Applying options for api1b debug1: Reading configuration data /etc/ssh_config debug1: Applying options for * debug1: Executing proxy command: exec ssh -C 10.0.0.2 nc 10.0.0.15 22 debug1: permanently_drop_suid: 501 debug1: identity file /Users/wi-fi/.ssh/id_rsa type 1 debug1: identity file /Users/wi-fi/.ssh/id_rsa.pub type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.9p1 Debian-5ubuntu1 debug1: match: OpenSSH_5.9p1 Debian-5ubuntu1 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.6 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr hmac-md5 debug1: kex: client->server aes128-ctr hmac-md5 debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host '10.0.0.15' is known and matches the RSA host key. debug1: Found key in /Users/wi-fi/.ssh/known_hosts:89637 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: Roaming not allowed by server debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey debug1: Next authentication method: publickey debug1: Offering RSA public key: /Users/wi-fi/.ssh/id_rsa debug1: Authentications that can continue: publickey debug1: No more authentication methods to try. Permission denied (publickey).
But what each of these publications fails to mention as a possible problem is that if your username doesn't match on both the intermediate and destination hosts, the ProxyCommand won't work until you specify the usernames in your ~/.ssh/config like this:
Host 10.0.0.15
HostName 10.0.0.15
User wi-fizzle
ProxyCommand ssh 10.0.0.2 nc %h %p
IdentityFile ~/.ssh/id_rsa
...
Reusing Connections
The transparent multi-hop connections can be very useful but you may find that it takes a second or two to establish each connection. This delay can become annoying if it happens a lot (e.g.: every time you save a file from the text editor).
The good news is that if you can configure SSH to reuse an existing connection. This means that for example if you have an SSH shell session running then a new connection for SCP can skip the connection setup phase. Two steps are required:
First, you must create a directory (or 'folder') which SSH will use to keep track of established connections:
mkdir ~/.ssh/tmp
ControlMaster auto ControlPath /home/YOUR-NAME/.ssh/tmp/%h_%p_%r
Howto: Enable Remote Desktop Admin on Windows 7 Home Premium
Howto: Enable Remote Desktop Admin on Windows 7 Home Premium
This worked fine for Win 7 x64: http://andrewblock.net/2010/02/23/enable-remote-desktop-on-windows-7-home-premium-64-32-bit/
Also: http://experts.windows.com/frms/windows_entertainment_and_connected_home/f/114/t/79427.aspx?PageIndex=28