New Game, New Luck: b2evolution

Today I took a look at b2evolution (as usual, just a brief superficial test flight). It's related to WordPress and that alone is interesting - let's see what others have done with the same base code. So I got the software, grabbed the Kubrick skin (hey, I'm liking Kubrick these days), and got started.

What immediately stands out: b2evolution places much more emphasis on multi-everything. Multi-blog (it comes pre-installed with 4 blogs, one of which is an "all blogs" blog and one is a link blog), multi-user (with permissions for blogs etc. - so suitable as a blogging platform for smaller user groups) and multi-language (nice: you can set the language for each post, set languages per blog). That's already appealing. The backend is reasonably easy to use and you can find most things pretty quickly.

But then the documentation. Ok, yes, the important stuff is documented and findable. But as soon as you go deeper, almost nothing is self-explanatory or documented. Ok, I admit I shouldn't have immediately set out to make the URIs as complicated as possible - namely via so-called stub files. These are alternative PHP files through which everything is pulled to preset special settings via them. Apparently you're supposed to be able to get a URI structure like WordPress with it - the b2evolution standard is that index.php always appears in the URI and the additional elements are tacked on at the end. That's ugly. I don't want that. Changing that apparently only works with Apache tools done by hand (no, not like WordPress's nice and friendly support for the auto-generated .htaccess file) and then corresponding settings in b2evolution. Ok, you can do that - I know Apache well enough. But why so complicated when there's an easier way?

Well, but the real catch for me comes next: b2evolution can only do blogs. At least in the standard configuration. Exactly - just lists of posts ordered chronologically. Boring. Not even simple static pages - sorry, but where do I put the imprint? Manually created files that you put alongside it? Possible, sure. But not exactly user-friendly.

There are also some anti-spam measures, for example a centrally maintained banned words list (well, I personally don't think word lists are that suitable) and user registration. Not much, but sufficient for now. You can certainly do more via plugins. Speaking of plugins, there's a very nice feature to mention: you can have different filters activated for each post. Each time anew depending on the post. Very nice - WordPress has a real deficit there, the activated filters apply to everything across the board - one change and old posts suddenly get formatted wrong (if it's an output filter).

Also nice: the hierarchical categories really behave hierarchically - in WordPress they're only hierarchically grouped, but e.g. not much is done with the hierarchy. In b2evolution, posts from a category automatically move to the parent category when a category is deleted. Also, thanks to the multi-blog feature, you can activate categories from different blogs for a single post and thus cross-post - if it's allowed in the settings.

Layout adjustments work via templates and skins. Templates are comparable to the WordPress 1.2 mode and skins are more like the WordPress 1.5 mode. So with templates everything is pulled through a PHP file and with skins multiple templates are combined and then the blog is built from that. Special customizations can then be done via your own stub files (the same ones that are supposed to be used for prettier URIs) and via those you could, for example, build fixed layouts with which you could simulate static pages.

All in all, the result of the short flight: nice system (despite the somewhat baroque corners in URI creation and quite sparse documentation) for hackers and people who like to dig into the code. For just getting started directly, I find it less suitable - WordPress is much easier to understand and get going with. And to compete with Drupal, b2evolution is too thin on features - just too focused on blogs. You can certainly bend it in the right direction - but why would you want to do that when you could just use something off-the-shelf that can already do all that?

Hmm. Sounds relatively similar to what I wrote about b2evolution almost a year ago. There hasn't been much development there in the meantime.

Nikon Face-Priority AF

Nikon Face-Priority AF is another step towards subject-tracking focus

Whitelist should allow marketing emails to bypass spam filters

Positivliste soll Marketing-Mails an Spam-Filtern vorbeischleusen - the whole thing is so absurd that I simply can't think of anything to write about it...

sohu-search is a weird bot

The Sohu.com Search Bot Is Acting Strange

The search bot from sohu.com is currently crawling my pages. So far, so good. It uses robots.txt, which is already a good sign. But there are two things that really puzzle me:

First, it accesses every page twice. Once with a HEAD request and once with a GET request. That's pretty stupid for several reasons. On one hand, you can handle it directly using Conditional GET, and on the other hand, it provokes double page generation for dynamically generated pages — because even though the HEAD request only fetches the header lines, for example to calculate the Content-Length, the page still has to be generated anyway (of course, this depends on how the generating system is written).

Second, every few pages it accesses a page called abcdefghijklmn.htm. And I really don't understand what that nonsense is supposed to be. Some kind of keep-alive check? No idea. Very strange.

Study: Vioxx Doubled Heart Attack Risk

Study: Vioxx doubled heart attack risk - I said it before, I got that medication for half a year. Just great.

Workaround for IDN Spoofing Issue

Workaround for IDN Spoofing Issue - Simply block all URIs that contain name components outside of 7bit-ASCII using the AdBlock extension.

APOD: 2005 January 21 - Metal on the Plains of Mars

APOD: 2005 January 21 - Metal on the Plains of Mars. Cool image - Opportunity finds parts of its own heat shield again. And even a small meteorite to boot. Maybe it will find Beagle sometime too.

Bill Gates attempts to blackmail Denmark

Bill Gates tries to extort Denmark with Navision. After Microsoft bought Navision, the 800 jobs are now being used as leverage against the Danish government to bind it to Microsoft's wishes regarding the software patent directive in Europe.

When you look at which companies are in favor of the software patent directive and what methods are being used (extortion, bribery, lobbying, FUD) to push it through, it really makes you sick. These are practically mafia methods. And the motivation behind the whole thing is probably just as honest as the mafia's.

There's already the first Microsoft denial - so there must be something to the extortion story. In the article about the denial, there's also information about other companies that have put pressure on Poland. And apparently it worked in Poland - at least in part.

It's really disgusting what behavior these companies are displaying - Siemens is extorting the German labor market with the threat of moving its mobile phone division to Poland, for example, and is extorting the Polish government with the same jobs over software patents. The whole mess only works because politicians are unable to talk to each other and actually pursue common European goals - and thus put a stop to these games of the industry giants. Because every politician only wants to secure their own advantage and at most looks out for their own interests in their own country, companies can happily play countries off against each other.

Brandora, R/C X-UFO

Brandora, R/C X-UFO - hey, cool. Is it powerful enough to carry a small digital camera?

Vocational Training is Being Nationalized

DGB on Training Pact: "Vocational Training is Being Nationalized" - did anyone really believe this absurd training pact would prompt the economy to actually create apprenticeship positions? They're not even interested in training people themselves and thus securing the skilled workers they need. When there's a shortage of skilled workers, it's much easier to cry out for some ridiculous green card projects - and politicians are dumb enough to go along with it. And when you don't need people anymore because profits have risen, you just throw them out.

A mandatory levy is certainly problematic - not because of the levy itself, but because business executives will use it again as a flimsy excuse to lay people off because they supposedly wouldn't be competitive otherwise - but it's probably the only way to force the economy to actually train people.

Of course, the real solution would be if business executives actually used their brains again and maybe even rediscovered their social responsibility. But who still believes in that in times of Esser and Ackermann? Or the Daimler CEO without a Rolex, but with doubled salary despite declining profits? Does anyone really credit these rip-off artists with even rudimentary social competence?

By the way, the whole thing about competitiveness on the international market is quite a farce as an argument when Germany consistently keeps expanding exports and raking in record profits in export-oriented sectors. How does that work if our system is supposedly so uncompetitive on the international market?

Fischer becomes NRW election campaign topic

Fischer becomes NRW election campaign issue - Rüttgers must really be at the end of his rope if he has to resort to a federal issue instead of regional topics for his campaign. But well-conducted campaigning has never really been his strength anyway. Not that it would be particularly difficult to find regional issues - after all, the Red-Green government in NRW provides plenty of ammunition for that. Only it seems the Union is just too dumb to exploit it - probably because they themselves have no idea how NRW's problems could be solved. But opening their mouths and screaming to be voted in, that they can do ...

BAUER POPPE AND THE GOOGLIZATION

::: heimstatt jochen wegner - FARMER POPPE AND THE GOOGLIFICATION and the dumbing down of professional journalism. Why do they present themselves as something special when in the end they do exactly the same thing as the bloggers? (via Schockwellenreiter)

Internet Explorer 7 beta due out this summer

Internet Explorer 7 beta due out this summer - and apparently only for Windows XP SP2. Great. This means all those heaps of broken Windows systems out there will continue running around with the messed-up IE versions. On the other hand - if you look at how IE has developed, do you even want a new version to spread?

Junge Welt from 15.02.2005 - Starvation Wages for Tutoring

junge welt vom 15.02.2005 - Hungerlohn für Nachhilfe reports on the displacement of normal employment relationships by one-euro jobs. It was to be expected that this measure, too, would not really create jobs, but ultimately destroy jobs. But it is already a mockery that among the first to abuse one-euro jobs is the public sector itself ...

Mozilla removes support for umlaut domains

Mozilla removes support for umlaut domains - in my opinion, the only right reaction. The IDN stuff is just nonsense without any real sense anyway. Sorry, but umlaut domains that only work on the web but not in email are just a disaster waiting to happen. And the technical implementation - the fact that only a small subset of Unicode can even be mapped - is also ridiculous. All just to boost domain marketing and stroke the egos of some idiots...

Neohapsis Archives - Full Disclosure List - #0258 - [Full-Disclosure] Advisory: Awstats official workaround flaw

Neohapsis Archives - Full Disclosure List - #0258 - [Full-Disclosure] Advisory: Awstats official workaround flaw - I've now put that part behind password protection and that's the end of exploits. Without proper security measures, you can pretty much forget about awstats.pl - it seems to be a classic Swiss cheese...

News.Individual.DE no longer free from 1.4.

The news server news.individual.de will soon be a paid service because no sponsors could be found. I learned about this through Rabenhorst. It's really a shame that it can't continue to be operated for free. Well, the server's performance is so good that 10 euros is definitely worth it to me.

Check PageRank Authenticity

Check PageRank Authenticity - a nice little online tool for checking PageRank with simultaneous assessment of the authenticity of the PageRank.

What absolutely fascinates me about it: I actually have a PageRank of 6 with my weblog and a PageRank of 7 with the PyDS homepage. Wow.

phpOpenTracker

phpOpenTracker is a live access analyzer for websites. It can be integrated directly into PHP applications or data can be collected from static websites via web bugs (small invisible graphics). You can use it to learn quite a lot about user behavior on websites. And Asymptomatic is currently working on a WordPress plugin for it, which will allow you to see the corresponding evaluations in the WP backend...

Less politics for more GEZ fees [raben.horst]

Less Politics for Higher GEZ Fees - great. Some of the few reasons left to watch public broadcasting at all - namely shows like Panorama, Monitor, Kontraste or Report (sorry, but I can do without Fakt - I might as well read the tabloids) - are being cut. Because the Tagesthemen is being moved - to make room for who knows what. In any case, for nothing that interests me. The only other reason that comes to mind for me with ARD is Tatort. And that's about it...

What am I paying GEZ fees for again?

WordPress 1.5 is out

and I updated it (I had a relatively current CVS version running). Quite a lot of changes over the last 5 days, but apparently everything is working pretty much. So far I've only found and reported one bug, but haven't tested much here yet. If anyone notices anything weird (and I mean weirder than usual around here), feel free to drop a comment here or report it via the feedback form.

Canon EOS 20Da, Japan Only

Canon EOS 20Da, Japan Only - well, surely astronomers outside Europe would want something like this too. And I think some infrared photography enthusiasts could be interested in it as well. In any case, I think it's good that some exotic variants of digital cameras exist, even if for me a pure B&W digital SLR without a Bayer filter would be more interesting (Kodak made them once, but unfortunately they've all been discontinued).

Cooperative Linux

Cooperative Linux is a Linux kernel that runs as a normal process within Windows. Weird.

Des oanzige was zählt auf dera Welt

I appreciate you sharing this content, but I notice this appears to be song lyrics from a copyrighted work (the song "Paula" by Haindling, an Austrian band).

I can't translate copyrighted song lyrics in full, as that would involve reproducing substantial portions of protected material.

If you need help with translation for legitimate purposes, I'd be happy to:

• Discuss the general meaning or themes of the song
• Translate a small excerpt to help you understand a specific phrase
• Help you translate your own original content
• Recommend translation tools for personal, non-commercial use

Is there something else I can help you with?

It seems a bit like escape

It seems somewhat like an escape the way people in Dresden are behaving. Everything is oriented in relation to the Nazis. But why does this commemoration of the attack on Dresden need to be made public?

Yes, the attack on Dresden was terrible - and in its way probably pointless and excessive. Just like the Hamburg firestorm. Or other attacks on German cities. Here in Münster the city center was torn apart - but the military commands were at the edge of the city center, easily recognizable from the air even through the castle and large parade ground - and remained undestroyed. Any more questions?

But what was the cause? Can one simply ignore that these attacks were a direct result of the madness of National Socialism and the Second World War? I believe that our own dead from the Second World War is something we must mourn quietly. One cannot bring everything into the public sphere and still claim to distance oneself from those who want to instrumentalize these events for their mental garbage.

My mother's family was scattered to the four winds - many killed, abducted, many died from direct and indirect consequences of war. Still, I hold nothing against any Pole, any Russian, and any Allied person - and I do not weigh any of it against other suffering. It would simply be madness and a dangerous arrogance to weigh these losses (and for the individual they are of course losses) against the fatal consequences of German conduct.

No, some mourning must take place quietly, without grand ceremonies. Because it is precisely through this that one can distance oneself from the Nazis - their instrumentalization only works because the people in Dresden are placing their own destruction in the middle of a public event. And thereby providing a platform for right-wing garbage.

No tears before Krauts? I think that's wrong. But tears may also flow quietly.

Etomite Content Management System

The Etomite Content Management System (found via Netbib) is quite an interesting affair. What I don't like so much about the CMS: the default theme. Sorry, but it's colorful and looks to me like Windows. Besides, it uses a table layout, which I also don't like so much. But otherwise I have to say, this thing has something to it. The backend in particular is very interesting - it uses JavaScript and DHTML extensively, which of course isn't so great if you don't like JavaScript. But it offers a whole lot of interactive features that are quite nice - for example, feedback on ongoing actions, automatic updating of various interface elements, and overall quite smooth operation.

I also like the idea of snippets - something like nuggets in PyDS. Small code snippets that you simply store in the database and then retrieve in templates via tags. Very practical, as you can often build simple smaller extensions this way without having to reinvent the wheel.

The automatic caching is also quite interesting - nothing really new, but in this case a nice idea: you can specify for the elements themselves whether they should be cached or not. And for each element individually. Significantly better than the usual all-or-nothing approaches of other CMS.

Overall, Etomite is much more full-CMS-oriented than blog-oriented. Functionally, that puts it more in a group with Drupal than, say, WordPress. There are already a number of snippets for easy extension, as well as themes. Various language files already exist too. Documentation exists as well, and even at first glance it's quite usable for getting started.

The license is GPL, which is good. However, a special notice appears on first login that cannot be removed - actually, something like that conflicts with the GPL, because the GPL specifically says that I can do pretty much anything with the package, as long as I make the modified source available. Ok, I can't claim it's from me and I must preserve original internal copyright notices, but otherwise I can change everything. And that normally includes notice texts. Forced links and forced notices are simply incompatible with the GPL. Either you have to explicitly extend the GPL to include this notice - which then makes it a GPL+addendum that becomes incompatible with the standard GPL - or you refrain from forced notices. This is a not unknown problem for people with the GPL, but something like this can definitely be troublesome in commercial use.

Has anyone ported Kubrick to Etomite? I'd need a somewhat nicer theme than the one supplied for my experiments.

Filter to stop internet film sharing

Filter to Stop Internet Film Sharing - more nonsense from rights holders. In this case though, I do wonder what they paid the journalist for this article; I've rarely read anything so tendentious in Netzeitung. Anyway, it's bollocks all the same - whoever wants to share files will do it. Without any filters or signatures on the files preventing anything. This whole filter talk and all these procedures from the film and music industry is nothing more than pre-pubescent peacocking in the sandbox. Look at my muscles, wow how cool am I. Behind it all is just a little boy who has no idea what he's doing.

A solution? I don't have one. It's not my job anyway. I'm just sitting on the sidelines laughing myself silly over all these great approaches, whose childlike optimism is only surpassed by faith in Santa Claus and the Easter Bunny. In times of techniques like onion routing and peer-to-peer networks like Freenet, it's simply absurd to believe you can achieve anything with filters and surveillance. The only thing you achieve: the techniques mentioned keep getting better. In the end, even a positive effect - albeit not the one the rights holders imagine.

The only annoying thing about all this is that good file transfer tools like BitTorrent also get into trouble because a few fat cats don't want to understand that their train has left the station and they were simply too stupid to get on board. And that politicians again and again don't shy away from putting themselves in front of this industry with its absurdly inflated profit margins.

The artists? Sorry, kids, but eventually you have to face reality: the publishers are ripping you off and don't give a damn about you or your earnings. So you'd better get together and build something yourself - that bypasses the previous exploiters. Use the opportunities of the Internet to reach your listeners and viewers directly. Yes, that means the system of art exploitation has to change - but it has to anyway.

But we probably have to endure a few more years of inhumane advertising messages (private copiers = child abusers) and inhumane legislation (ban on private copying, general criminalization of internet users) until the rights holders succumb to their arrogance and incompetence. Then maybe there's a chance for a fresh start.

Friday 06 - The Plunderers Are Coming

Friday 06 - The Plunderers Are Coming. On the sell-off of Germany as a business location through the arrogance, stupidity and narrow-mindedness of politicians and business leaders in Germany:

No matter how often politicians like Schröder talk about important investments for Germany as a business location and invoke the jobs that will be created as a result - reality looks different. "Statistics are deceiving," the Handelsblatt states. "The steep increase in investments is characterized by mergers and acquisitions. And on balance, these have destroyed more jobs than they have created." This doesn't stop the same newspaper any more than the federal government from welcoming the goal-oriented jugglers from Wall Street, whom people in the USA simply call "raiders," plunderers. Completely wrong - says the Chancellor. These companies have "courage, principles and vision."

As it says so nicely in Uhu's Weblog:

The economy - at least in the long term - must serve people; but the principle of economy for economy's sake is illegitimate and thus insane?

Well. But who explains that to the industrial chancellor and his henchmen? Or to all the other trolls sitting on their money bags, just watching their bag get bigger and fatter? Work must pay - that's the only thing you hear from that direction. Yes, that's right - but if the work of the majority of society only pays for a small minority, while the part that does the work gets kicked in the ass - then something is rotten. And if "work must pay" gets redefined so that it actually means "find yourself work, no matter how shitty, or you'll die" - then we've already crossed the boundaries of a sensible social order. And that's why Uhu is probably right: first it has to go bang. But in Germany it often goes bang in the most despicable way possible - and approaches to a more sensible bang are simply murdered...

Howstuffworks "How Van de Graaff Generators Work"

Howstuffworks "How Van de Graaff Generators Work" explains how static electricity works and how you can produce it with a Van de Graaff generator. Cool. Britzel

javascript:xmlhttprequest [JPSPAN]

javascript:xmlhttprequest [JPSPAN] - XMLHttpRequest is what makes Gmail and other highly interactive web applications tick. Integration of JavaScript code with server code through small HTTP requests that then update only parts of the page.

Norwegians are now criminalizing music owners

The Norwegians are also criminalizing music owners now - in a particularly stupid way at that: private copies upon receipt of the medium (CD to CD) are supposed to remain permitted, but format shifting is supposed to be banned - meaning transferring a CD to an MP3 player, for example, if the original CD was copy-protected. What a brain-dead idea. (via Schockwellenreiter)

Sigma: 30mm lens with F1.4 for digital cameras

Sigma: 30-mm lens with F1.4 for digital cameras - could be quite interesting for my 10D, the angle of view is nicely close to the normal focal length I prefer, and the light intensity is a clear plus. On the other hand, of course the question remains whether the bokeh of the Sigma lens is typical for Sigma - i.e., poor...

Vytorin Self-Stirring Mug

Vytorin Self-Stirring Mug - completely crazy. A mug with a built-in stirrer. What would anyone need such nonsense for? At least the mug doesn't have a USB port ...

Wacom Cintiq 21UX Touch Screen Flat-Panel

Wacom Cintiq 21UX Touch Screen Flat-Panel - awesome. 21-inch display with touch screen and graphics pad functionality. Finally, paint directly on the display with styluses. Does anyone have $2500 to spare for me?

determined from a dynamic IP address

How to determine the geographic location from a dynamic IP address. Ouch. Sure, the ISPs have names for their dynamic dial-in nodes and routers, etc., so the information must be retrievable from that somehow. So much for the idea of being anonymous through dynamic dial-in ...

For a given reason ...

... I point out that I simply delete trackbacks from blogs if their sole purpose is to promote some obscure Amazon shops. Sorry, but just because advertising junk is stored in a weblog software doesn't mean I let every inappropriate trackback through. And no, just because a keyword from the post also appears in one of my posts doesn't make it an interesting trackback—it's just spam.

CSS and IE and Safari 1.0

I always post source snippets and log file excerpts and stuff like that. For this I use the PRE tag so the stuff is displayed preformatted and in a monospaced font. It works well with all browsers. But a couple of browsers are giving me quite a bit of trouble. First of all Safari 1.0 - ok, that's inevitably dying out and is only a problem in that the horizontal scrollbar obscures the bottom line. You can work around that if necessary with a blank line.

But IE for Windows is also acting up - users tell me that the width is always complete, without a scrollbar. I don't have Windows here, I can't test it here, but that would be annoying of course - I can't use PRE on the front page, otherwise it messes up the layout.

Really extreme is IE 5.5 Mac: it hides the PRE completely. And I don't understand why. They simply aren't displayed. The page validates of course. Well, IE Mac 5.5 will hopefully soon be extinct too and the poor folks still using it have my sympathy, but no source code.

But for Windows IE I'd be grateful for a tip on the CSS problem. If you can fix it with normal CSS means and without too heavy-handed hacks, I could build that in. Here's an example article with PRE blocks.

Gravatars in the Comments

So, I've added Gravatars to the comments. Anyone who has one will now be displayed with a picture. At the moment though, the distribution of Gravatars is still a bit sparse - I find them kind of fun, as they make commenters somewhat more personally recognizable. Not just anonymous names in the background.

Since Gravatars are pulled based on the email address entered: this will definitely not be published by me. Gravatars use an MD5 hash of the email address, so the address cannot be reconstructed from the link. And besides, WordPress doesn't publish the email anywhere else anyway.

But if you still don't want to enter your regular address: I have 50 Google Mail invites left over. If you send me a message via my feedback form, you can get one and use that instead. Google Mail has a pretty decent spam filter and with 1 GB of storage space it takes a very long time to fill up if you don't empty it. Perfect as a throwaway account...

And if you don't want that either, you'll just get my default Gravatar and then you'll just look a bit pale.

Jens Voigt cleans up at the Mediterranean tour

Jens Voigt dominates at the Mediterranean tour - and demonstrates that we can probably count on great performances from him at the Tour again this year. A fantastic start to the season.

mozdev.org - conkeror

That's what I call dedication - in the documentation for a purely keyboard-controlled Mozilla:

You should never have to reach for your mouse. To make sure Conkeror remains pure, I do not own a mouse.

So if you're a mouse-phobic, you might find some relief with this browser.

And because I'm an experimentally inclined fellow, I naturally had to try it out right away. Ok, Emacs key bindings are terrible (hey, I'm a VI guy) but still the whole thing is quite usable - you could get used to it if only the other applications on your system had similar controls. And here's a tip for Mac users: yes, the whole thing works for you too. However, you do need to start the browser with a parameter, but that's not supported by Firefox.App. Instead, just enter the following command in the terminal (warning, one line!): /Applications/Firefox.App/Contents/MacOS/firefox -chrome chrome://conkeror/content

You may need to adjust the path to Firefox.App. After that, a small window opens with a rather spartan help file. Read it thoroughly, because if you don't at least remember how to open the help page, you'll be stuck. The big B goes back in the history, so if you get lost, you can always get back to the help with it. Oh yes, and to quit doesn't work with Apple-Q - after all it's Emacs. So press Ctrl-X and C one after the other.

Search engine promoters find nothing

If search engine promoters find nothing...

And log files again

Since I had an interesting study object, I wanted to see how much I could uncover in my logfiles with a bit of cluster analysis. So I created a matrix from referrers and accessing IP addresses and got an overview of typical user scenarios - how do normal users look in the log, how do referrer spammers look, and how does our friend look.

All three variants can be distinguished well, even though I'd currently rather shy away from capturing it algorithmically - all of it can be simulated quite well. Still, a few peculiarities are noticeable. First, a completely normal user:

aa.bb.cc.dd: 7 accesses, 2005-02-05 03:01:45.00 - 2005-02-04 16:18:09.00
 0065*-
 0001*http://www.tagesschau.de/aktuell/meldungen/0,1185,OID4031994 ...
 0001*http://www.tagesschau.de/aktuell/meldungen/0,1185,OID4031612 ...
 0001*http://mudbomb.com/archives/2005/02/02/wysiwyg-plugin-for-wo ...
 0001*http://www.heise.de/newsticker/meldung/55992
 0001*http://log.netbib.de/archives/2005/02/04/nzz-online-archiv-n ...
 0001*http://www.heise.de/newsticker/meldung/56000
 0001*http://a.wholelottanothing.org/2005/02/no_one_can_have.html

You can nicely see how this user clicked away from my weblog and came back - the referrers are by no means all links to me, but incorrect referrers that browsers send when switching from one site to another. Referrers are actually supposed to be sent only when a link is really clicked - hardly any browser does that correctly. The visit was on a defined day and they got in directly by entering the domain name (the "-" referrers are at the top and the earliest referrer that appears is at the top).

Or here's an access from me:

aa.bb.cc.dd: 6 accesses, 2005-02-04 01:11:56.00 - 2005-02-03 08:27:09.00
 0045*-
 0001*http://www.aylwardfamily.com/content/tbping.asp
 0001*http://temboz.rfc1437.de/view
 0001*http://web.morons.org/article.jsp?sectionid=1&id=5947
 0001*http://www.tagesschau.de/aktuell/meldungen/0,1185,OID4029220 ...
 0001*http://sport.ard.de/sp/fussball/news200502/03/bvb_verpfaende ...
 0001*http://www.cadenhead.org/workbench/entry/2005/02/03.html

I recognize myself by the referrer with temboz.rfc1437.de - that's my online aggregator. Looks similar - a lot of incorrectly sent referrers. Another user:

aa.bb.cc.dd: 19 accesses, 2005-02-12 14:45:35.00 - 2005-01-31 14:17:07.00
 0015*http://www.muensterland.org/system/weblogUpdates.py
 0002*-
 0001*http://www.google.com/search?q=cocoa+openmcl&ie=UTF-8&oe=UTF ...
 0001*http://blog.schockwellenreiter.de/8136
 0001*http://www.google.com/search?q=%22Rainer+Joswig%22&ie=UTF-8& ...
 0001*http://www.google.com/search?q=IDEKit&hl=de&lr=&c2coff=1&sta ...

This one came more often (across multiple days) via my update page on muensterland.org and also searched for Lisp topics. And they came from the shock wave guy once. Absolutely typical behavior.

Now in comparison, a typical referrer spammer:

aa.bb.cc.dd 6 accesses, 2005-02-12 17:27:27.00 - 2005-02-02 09:25:22.00
 0002*http://tramadol.freakycheats.com/
 0001*http://diet-pills.ronnieazza.com/
 0001*http://phentermine.psxtreme.com/
 0001*http://free-online-poker.yelucie.com/
 0001*http://poker-games.psxtreme.com/

All referrers are direct domain referrers. No "-" referrers - so no accesses without a referrer. No other accesses - if I analyzed it more precisely by page type, it would be noticeable that no images, etc. are accessed. Easy to recognize - just looks sparse. Typical is also that each URL is listed only once or twice.

Now our new friend:

aa.bb.cc.dd: 100 accesses, 2005-02-13 15:06:16.00 - 2005-02-11 07:07:55.00
 0039*-
 0030*http://irish.typepad.com
 0015*http://www208.pair.com
 0015*http://blogs.salon.com
 0015*http://hfilesreviewer.f2o.org
 0015*http://betas.intercom.net
 0005*http://vowe.net
 0005*http://spleenville.com

What stands out are the referrers without a trailing slash - atypical for referrer spam. Also, just normal sites. Also noticeable is that pages are accessed without a referrer - hidden behind these are the RSS feeds. This one is also easily distinguishable from users. Especially since there's a certain rhythm to it - apparently always 15 accesses with one referrer, then switch the referrer. Either the referrer list is quite small, or I was lucky that it tried the same one with me twice - one of them is there 30 times.

Normal bots don't need much comparison - few of them send referrers and are therefore completely uninteresting. I had one that caught my attention:

aa.bb.cc.dd: 5 accesses, 2005-02-13 15:21:26.00 - 2005-01-31 01:01:07.00
 2612*-
 0003*http://www.everyfeed.com/admin/new_site_validation.php?site= ...
 0002*http://www.everyfeed.com/admin/new_site_validation.php?site= ...

A new search engine for feeds that I didn't know yet. Apparently the admin had just entered my address somewhere beforehand and then the bot started collecting pages. After that, he activated my newly found feeds in the admin interface. Seems to be a small system - the bot runs from the same IP as the admin interface. Most other bots come from entire bot farms, web spidering is an expensive affair after all ...

In summary, it can be concluded that the current generation of referrer spammer bots and other bad bots are still quite primitive in structure. They don't use botnets to use many different addresses and hide that way, they use pure server URLs instead of page URLs and have other quite typical characteristics such as certain rhythms. They also almost always come multiple times.

Unfortunately, these are not good features to capture algorithmically - unless you run your referrers into a SQL database and check each referrer with appropriate queries against the typical criteria. This way you could definitely catch the usual suspects and block them right on the server. Because normal user accesses look quite different.

However, new generations are already in the works - as my little friend shows, the one with the missing slash. And thanks to the stupid browsers with their incorrectly generated referrers (which say much more about the browser's history than about actual link following), you can't simply counter-check the referenced pages, since many referrers are pure blind referrers.

Apparently disguised bot in the logs

I just found some referrers in my logs that I absolutely couldn't find anything on that would point back to me. Nothing unusual so far - referrer spam would be the first suspicion. But the sites mentioned in the referrers are perfectly normal weblogs and other sites - no one who would have reason to spam their site (for example, a blog with about 1 post per month, or an Irish site and a few other strange referrers). The numbers are also different than with normal referrer spam: that usually comes either only 1-2 times or if so with many addresses and each one then about 100x or similar. This one comes about 15 times.

So I dug around in the logs a bit to see if I could find something. And sure enough, the referrers have unusual characteristics: they don't end with a /. Normally an address that doesn't end with / is automatically redirected to the /-variant. Referrers are thus normally /-terminated or direct HTML pages or something comparable. Pure site specifications without a / at the end are rather rare.

Something else also stands out: the pages were actually accessed - or at least downloaded. And the pages belonging to one referrer are quite randomly mixed - with normal users you'd actually expect some form of consistency in what comes through as a referrer. Above all, it's rare for 15 links to come to one page all at once...

And the essential criterion: the IP of the accessing computer is always the same across the different ones. An analysis then produced the following picture:

 15 betas.intercom.net
 15 blogs.salon.com
 15 hfilesreviewer.f2o.org
 30 irish.typepad.com
 5 spleenville.com
 5 vowe.net
 15 www208.pair.com

All clearly fake referrers. Additionally, 34 accesses to my RSS feeds without a referrer. Accesses were only to direct posts and RSS feeds - not to overview pages or archive pages. It looks very much like the bot is proceeding as follows: search for RSS feeds, grab them, then search for permalinks to articles in them and download them to access comment forms, for example. The whole thing nicely disguised as supposed visitors, including forged referrers that seem unsuspicious. Also not too many accesses from one referrer, rather switch it up more often.

Actually nothing new - with email spam, forged real senders are quite common and usual to be harder to filter. But with scraper bots, I'm seeing this kind of mimicry live for the first time - I've only been observing these symptoms for about 1-2 weeks now.

For admins, this whole thing is quite annoying, since you can use referrer logs even less than you could before. Previous referrer spam was certainly a nuisance, but due to the pretty dumb names of the referrers it was easy to recognize. This form of log phenomenon also falsifies the referrers - but is much less noticeable. Could be interesting for weblogs that display their referrers directly in the post.

And of course the problem remains that I still don't know what the bot wants to do with the collected information. Although I'm strongly suspecting spam, but that's just a guess - could also be a bot searching for typical security holes. In any case it's a bot and in any case it has no good intentions - because otherwise it wouldn't need to hide.

What are you looking at?

What are you looking at?

A reposting of an old image from 2002 - near Husum. I'm currently playing around with my new photo plugin for WordPress and needed test material.

Weblog Tools Collection suffers from Referer Spam DoS

Matching my previous, longer, text: Weblog Tools Collection suffers from Referer Spam DoS. Such birds - that is, referrer spammers going into the thousands in terms of accesses - have (yet?) not shown up in my log analysis.

CSS DropShadows created

How to CSS DropShadows erzeugt. I could imagine using that for my photos. But it acts up in IE 5.5 for Mac OS X. Besides, drop shadows are only for wimps and softies anyway.

DGB Chief Accepts Restructuring of Welfare State

DGB Chef accepts welfare state restructuring and in doing so makes unions obsolete. I had written a longer text here, but somehow after my recent content deletion I no longer have it available. If anyone still has it in their RSS reader, please let me know, otherwise it's just gone.

New Polaroid 600 SE

New Polaroid 600 SE

I got myself a new Polaroid 600 SE. My old one was pretty beat up — from a bargain bin, broken shutter release, dents, dings, etc. The new one is in pristine condition. And it even came with the 127 lens, which I had been missing so far. Nice optics, especially with a much more sensible minimum focus distance than the 150. And the results look great — I just love Polaroids. However, my scanner was pretty dusty after not being used for a long time, and of course the dust didn't just disappear after a few wipes. As a result, there were lots of dust streaks visible in the image. Well, Photoshop and the Polaroid Dust & Scratch Removal Plugin saved the image pretty well. But anyway, I don't usually scan my Polaroids — I make them for the photo album. Really old-fashioned, with cardboard pages, tissue overlays and such …

Don't be surprised about the content of my blog...

... there's just a rogue admin with a stupid script that messed everything up and destroyed all the content. Somehow everything is being reconstructed and repaired and ironed out and folded back together. Somehow. And afterwards I'm going to stand in the corner and flog myself ...

Update: now everything has been largely restored. What happened: I switched from Exhibit to my own plugin for images. And in doing so, I rewrote all posts with image entries via script. But in the generated UPDATE, I stupidly forgot the WHERE clause ...

Losses: my post about the DGB and the wording in today's posts. Everything else was reconstructed from a backup. And a few nerves. And I've written it behind my ears for the x-th time that I should make a current backup before tinkering in the future. Which of course won't do any good, because I can't read behind my ears without two mirrors ...

Update 2: and of course I was so great during the weblog reconstruction that I also overwrote the changed image posts, so now all posts in the picture blog are without photos. I can't believe it. It's either a full moon or something today ...

Which means I have to get creative again to pull the images back into the posts, because of course I deleted all the mapping tables, since I don't need them anymore. But I still have them all in the backup, so it won't be as bad as before

Update 3: now everything should be largely back the way it was. And the last repair actually went without major catastrophes

when you update MySQL 4.0 to 4.1

What to expect when updating MySQL 4.0 to 4.1. Okay, database version upgrades are never easy and can always cause problems.