software/misc
Miscellaneous Software Stuff
Log Analysis
Regular Expressions
http://code.google.com/p/semicomplete/wiki/GrokDiscovery
Compression for Executables
UPX: Ultimate Packer for Executables http://upx.sourceforge.net/
Webmail
the null webmail client http://www.nulllogic.ca/webmail/
Standard ML
The SML Standard Basis (of types)
Mutually recursive function using 'and' syntax:
fun foo 0 = "beer car" | foo n = bar(n-1) and bar 0 = "bacon" | bar n = foo(n-1);
Network programming in SML/NJ: https://github.com/tonyg/smlnj-networking/
plan9 and Inferno
"Styx-on-a-Brick": networked LEGO device http://doc.cat-v.org/inferno/4th_edition/styx-on-a-brick/
plan9 on a GuruPlug http://plan9.bell-labs.com/wiki/plan9/guruplug/index.html
NoSQL
NoSQL Data Modeling Techniques http://highlyscalable.wordpress.com/2012/03/01/nosql-data-modeling-techniques/
PHP Problems
[PHP: A fractal of bad design](http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/)
[PHP Sucks but it doesn't matter](http://www.codinghorror.com/blog/2008/05/php-sucks-but-it-doesnt-matter.html)
LISP Visualization
Nested boxes instead of nested parens: http://pretty-lisp.org/
Webmail
Squirrel mail is ok, CiderWebmail looks better. Have had problems deploying Alpine Webmail.
Structured Wikis
Circa May 2012 there are a few options for structured data in wikis:
[Semantic MediaWiki](https://en.wikipedia.org/wiki/Semantic_MediaWiki) is probably the best option, though it is unfortunately in PHP. This is a series of plugins/extensions to vanilla MediaWiki, which add editing (with or without forms), querying, visualization, import/export, etc.
MediaWiki with the [WikiDB](http://www.mediawiki.org/wiki/Extension:WikiDB) (PHP)
DokuWiki with [data plugin](http://www.dokuwiki.org/plugin:data) (PHP)
eLog could be used to collaboratively edit simple data.
TWiki has structured data features and an input form (Perl).
DBPedia uses [custom extraction tools](http://wiki.dbpedia.org/Documentation?v=6hz) for MediaWiki written in Scala. dbpedialite uses just calls the Wikipedia API (Ruby).
You can [search](http://www.wikimatrix.org/search.php?sid=54395) wikimatrix for wiki engines which have a "structured data" feature.
Semantic Data Publishing
[OntoWiki](http://ontowiki.net/Projects/OntoWiki/Features)
[Google Refine](https://en.wikipedia.org/wiki/Google_Refine) (formerly known as Freebase Gridworks) seems pretty useful for "cleaning up" datasets. It's a Java "workstation" app with a web/http interface.
USB Performance Issues
Parallelized mass storage transfers are much slower than serialized: http://blog.dinkevich.com/why-usb-transfers-are-slow/
Inkscape Black Rectangle Box Issue
For some reason Inkscape sometimes inserts a black box into .svg files which are invisible within Inkscape but visible when exported.
To remove this box, edit the raw XML using the XML Editor in Inkscape (Shift-Ctrl-X), look for an svg:flowRoot element within the first layer, and delete it.
Latency Numbers
From https://gist.github.com/2841832:
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000 ns
Send 1K bytes over 1 Gbps network 10,000 ns 0.01 ms
Read 4K randomly from SSD* 150,000 ns 0.15 ms
Read 1 MB sequentially from memory 250,000 ns 0.25 ms
Round trip within same datacenter 500,000 ns 0.5 ms
Read 1 MB sequentially from SSD* 1,000,000 ns 1 ms 4X memory
Disk seek 10,000,000 ns 10 ms 20x datacenter roundtrip
Read 1 MB sequentially from disk 20,000,000 ns 20 ms 80x memory, 20X SSD
Send packet CA->Netherlands->CA 150,000,000 ns 150 ms
Notes:
1 ns = 10-9 seconds
1 ms = 10-3 seconds
* Assuming ~1GB/sec SSD
Great Hacks
HTML as image: http://lcamtuf.coredump.cx/squirrel/
xscreensaver
To lock the screen immediately, do:
xscreensaver-command -lock
Lua Tools
http://haserl.sourceforge.net/ "Dynamic web content in 20K"
Semantic Wiki Setup Notes
There is a [](Semantic MediaWiki quick reference), as well as good documentation of the [http://www.mediawiki.org/wiki/Extension:Semantic_Forms/Quick_start_guide](Semantic Forms) plugin.
[http://www.mediawiki.org/wiki/Extension:External_Data](External Data) plugin allows remote import of data via HTTP API call (cached) or file upload (eg, XML, csv).
"The djb Way"
[Dan Bernstein](http://cr.yp.to/djb.html) writes a lot of network daemons in a unique style. Eg, daemontools, ucspi-tcp, djbdns, qmail, and publicfile.
- [The djb way](http://thedjbway.b0llix.net/readme.html)
How TTYs Work
<http://www.linusakesson.net/programming/tty/>
Covers nohup
, piping, process groups, session leaders, TTYs, PTYs, stty
, etc.
Web Security
<http://stackoverflow.com/questions/549/the-definitive-guide-to-forms-based-website-authentication>
<http://code.google.com/edu/security/index.html>
<https://www.owasp.org/index.php/Main_Page>
Capture all HTTP Traffic to File
Install tcpflow
and do:
$ mkdir ~/full_dump; cd ~/full_dump $ sudo tcpflow -i wlan0 -b 2097152 tcp port 80 or tcp port 443
Saves up to 2MB
wget
For recursive, I usually want something like:
wget -r -l 3 <baseurl>
To just get files, ignoring structure:
wget -r -l 3 -nd <baseurl>
QEMU with chroot
Manual way:
$ sudo chroot ./rootfs-dir/ qemu-arm-static /usr/bin/env
The automatic way is to copy the host's qemu-arm-static to usr/bin in the chroot, then just run the command and qemu-arm-static will be used automagically:
$ sudo chroot ./rootfs-dir/ /usr/bin/env
Starting an SSL CA
"[cryptography] How much does it cost to start a root CA ?" Jon Callas <jon at callas.org>, Sat Jan 5 14:42:31 EST 2013 http://lists.randombit.net/pipermail/cryptography/2013-January/003601.html
Answer: $250k over 14 months, 40% hardware, 60% people