You are viewing wikitech

Wikimedia technical discussions' Journal
 
[Most Recent Entries] [Calendar View] [Friends]

Below are the 20 most recent journal entries recorded in Wikimedia technical discussions' LiveJournal:

    [ << Previous 20 ]
    Friday, November 3rd, 2006
    10:52 am
    [tekanji]
    Templates Explained?
    I'm interested in learning how to create complex templates for my wiki, but I don't find Wikimedia's Template FAQ useful at all. Does anyone know of somewhere that has a more user-friendly explanation of template creation?
    Thursday, September 7th, 2006
    2:56 pm
    [nikor]
    Squid replacement?
    Hey!
    Just wanted to point at varnish.
    I don't think they have had an official release yet but it looks like there is a lot of promise.

    sourceforge: http://sourceforge.net/projects/varnish/
    homepage: http://varnish.projects.linpro.no/
    Friday, March 31st, 2006
    9:58 pm
    [ice_is_blue]
    I Broke It Somehow...
    I've been a happy user of an offline (personal use w/ EasyPHP) MediaWiki for several months, but in the past week I've been having a problem. I can make edits and see those edits on the edit page... but the actual page itself doesn't update. The edits are saved. I can shut the machine down, reboot, and then see the changes on the edit page. But the actual article page won't update.

    My setup:
    WinXP Home SP2
    Firefox (also tried in IE)
    (according to the Special:Version page...)
    MediaWiki: 1.5.6
    PHP: 4.3.10 (apache)
    MySQL: 4.1.9-max

    I've tried stopping and restarting EasyPHP (PHP/MySQL), dumping the internet cache, and the thing of it is.... I don't recall making any changes to my system.

    Any suggestions would be welcome. (Or if there's a better forum for asking this question...) Thanks.

    ~ Isis

    Current Mood: frustrated
    Monday, February 13th, 2006
    8:12 pm
    [vampwillow]
    Wikimedia Projects
    For those who weren't aware, one of the Florida servers fell over this morning - apparently due to physcial hard disk failure - which caused onward problems throughout the server farm resulting in a need for multitudes of reboots and relinking. This happened around 11pm GMT and techs from west coast and australia (amongst others) were up during the middle of their nights to sort it. Many of the issues have been sorted but there are still some on-going problems, as such no wikipedia or wikimedia project should be considered as stable for the moment.
    Saturday, February 11th, 2006
    12:11 pm
    [lithiana]
    initial thoughts on the T2000

    so we received a T2000 for evaluation a couple of days ago. i spent yesterday fiddling with this and running some benchmarks.

    first thoughts:

    • it shipped with Solaris 10 3/05 HW2 and the Java Enterprise System pre-installed, rather than the current 1/06 release. it wasn’t hard to upgrade it to 1/06 remotely, but i found this a bit surprising.
    • the disk was partitioned with Sun’s standard pre-installed scheme, which left 50GB of the first disk and the entirety of the second disk available. setting this up as a mirrored disk was quite simple
    • the ALOM - remote management - is very nice. you can connect to it via a serial console or the network, power cycle the machine, connect to its console, and a few other things. the SPARC OpenBoot PROM is much more suited than a PC BIOS to this kind of remote management, since it’s controlled entirely via a command line. in general, managing the server was very pleasant.

    more importantly, though, how does it perform? from benchmarks seen elsewhere, i was expecting the T2000 to be as fast as, or faster than, our existing dual-Opteron systems - the 8-core, 1.2GHz model is reported to be comparable to or faster than a 2-CPU dual core Opteron for web serving workloads. ours is a 6-core, 1.0GHz version, so this seems like a reasonable expectation.

    i started by installing our standard Apache software on the machine. software versions are as follows:

    SunOS hydra.pmtpa.wmnet 5.10 Generic_118822-27 sun4v sparc SUNW,Sun-Fire-T200
    Server version: Apache/1.3.33 (Unix)
    PHP 5.1.2 (cli) (built: Feb 10 2006 17:26:41)
    /opt/mysql/mysql/bin/mysql  Ver 14.12 Distrib 5.0.18, for sun-solaris2.10 (sparc) using readline 5.0
    

    i compared this to a standard Linux host, srv54:

    Linux srv54 2.6.9-1.667smp #1 SMP Tue Nov 2 15:09:11 EST 2004 x86_64 x86_64 x86_64 GNU/Linux
    Server version: Apache/1.3.33 (Unix)
    PHP 5.1.1 (cli) (built: Jan 13 2006 08:11:22)
    mysql  Ver 11.18 Distrib 3.23.58, for redhat-linux-gnu (x86_64)
    

    i then than “ab” against both machines (running MediaWiki, our PHP web application), using an increasing number of threads. the results were as follows:

    threads req/s (Hydra) time/req (Hydra) req/s (srv54) time/req (srv54)
    1 2.34 455 ms 9.38 100ms
    2 4.84 208 ms 17.84 54 ms
    4 8.87 130 ms 25.05 37ms
    6 13.7 71 ms 29.05 33 ms
    8 16.5 59 ms 27.45 40 ms
    12 20.32 47 ms 29.27 33 ms
    16 19.98 48 ms 29.79 33 ms
    24 19.29 53 ms 29.16 33 ms

    hmm. the T2000 seems to be almost a third slower than the Opteron system. after this, i wondered whether the difference was caused by some network issues, rather than performance issues, so i made a simple, cpu-bound benchmark:

    <?php 
      for ($i = 0; $i < 100000; $i++) {
        $n += $i * (int)($i / 2);
      }
      print $n;
    ?>
    

    the T2000 consistently served this at 40 req/s, while the Opteron managed 60 req/s. so, about a third slower seems a reasonable result.

    this is obviously a very rough benchmark, but i don’t think it too likely that more fiddling will produce a more impressive result (if anyone has any suggestions, though, please let me know!)

    is it worth buying, then? well, a 6-core, 1.0GHz T1000 with 2GB RAM costs $2,995 list price, while the dual Opteron system with 4GB costs $2,613. an 8-core T1000 with 8GB is $6,995. a T2000 is even more expensive ($12.395 for 8 cores, 8GB). so, the T1000 isn’t too far off - and a 2/4GB, 8-core model might be even better, if i could find a price - but it’s still a bit more expensive. Sun claims that the T1 CPU saves money on pack space and power rather than initial cost. well, i haven’t managed to measure how much power the T2000 is using yet, but i’ll see if i can get our colo person to do that at some point soon. i think it’ll have to save a lot of power to make it a worthwhile purchase, though.

    update (2006-02-28):

    so based on Sun's SPECweb2005 submission for the T2000, i made some changes to /etc/system:

    set rlim_fd_max=500000
    set rlim_fd_cur=500000
    set sq_max_size=0
    set ipge:ipge_bcopy_thresh = 1536
    set ipge:ipge_dvma_thresh = 1536
    set ipge:ipge_tx_syncq=1
    set consistent_coloring=2
    

    and IP parameters:

    psradm -i 1 2 3 5 6 7 9 10 11 13 14 15 17 18 19 21 22 23
    ndd -set /dev/tcp tcp_smallest_anon_port 2048
    ndd -set /dev/tcp tcp_conn_req_max_q 102400
    ndd -set /dev/tcp tcp_conn_req_max_q0 102400
    ndd -set /dev/tcp tcp_max_buf 4194304
    ndd -set /dev/tcp tcp_cwnd_max 2097152
    ndd -set /dev/tcp tcp_recv_hiwat 400000
    ndd -set /dev/tcp tcp_xmit_hiwat 400000
    ndd -set /dev/ipge instance 1,2,3,4,5,6,7,8,9
    ndd -set /dev/ipge rx_intr_pkts 600
    ndd -set /dev/ipge rx_intr_time 600
    

    after this, the T2000 serves Special:Version at 29.75 requests/sec, vs 28.21 requests/sec for the Opteron system (at a concurrency of 40). so, it's now slightly faster, but still quite a bit more expensive: assuming an 8-core T1000 can process 38.65 requests/sec, it works out to $180/req/s, vs $92/req/s for the Opteron. a 6-core T1000 would be $100.67/req/s - close, but not quite there.

    Tim's also run some more benchmarks focusing more on raw CPU power than transactional speed.

    update (2007-06-21):

    frostcs mentioned below that performance for this kind of PHP application might have improved a lot recently. the impression we got from talking to Sun at the time was that PHP5 had a particular problem with concurrency that caused the poor performance. unfortunately, we don't have this machine anymore, but i'd be interested to see the results if anyone else can repeat the benchmark.

    Friday, February 10th, 2006
    1:35 am
    [brionv]
    PHP references must die
    Debugging an oddity in a new special page, I found this gem of a behavior... using array references is not such a hot idea. Take this simple case, of an outer loop and an inner loop iterating over the same array:

    echo "Nested iteration over a short array:\n";
    $set = range( 1, 3 );
    foreach( $set as $i ) {
    	echo "- $i\n";
    	foreach( $set as $j ) {
    		echo " -> $i, $j\n";
    	}
    }


    This works just like you'd expect:

    - 1
     -> 1, 1
     -> 1, 2
     -> 1, 3
    - 2
     -> 2, 1
     -> 2, 2
     -> 2, 3
    - 3
     -> 3, 1
     -> 3, 2
     -> 3, 3


    But here's the case my code was encountering:

    echo "\nSame thing, but the variable was set as a reference:\n";
    $ref =& $set;
    foreach( $ref as $i ) {
    	echo "- $i\n";
    	foreach( $ref as $j ) {
    		echo " -> $i, $j\n";
    	}
    }


    which renders:

    Same thing, but the variable was set as a reference:
    - 1
     -> 1, 1
     -> 1, 2
     -> 1, 3


    Apparently if the thing you're iterating over happens to have been set as a reference, it happily stomps over the array cursors, and the outer loop ends after the first iteration of the inner loop. Nice!

    Really, references in PHP just plain suck. At least with PHP5 we'll have the chance to get rid of most of them, since object assignment will work as you'd expect instead of a copy...

    HOWEVER!

    The real ugly part comes when you realize that the "global" statement actually works via hidden references:

    function iter() {
    	echo "\nSame thing, but the variable is a global:\n";
    	global $set;
    	foreach( $set as $i ) {
    		echo "- $i\n";
    		foreach( $set as $j ) {
    			echo " -> $i, $j\n";
    		}
    	}
    }


    Since global $set; is equivalent to something like $set =& $GLOBALS['set'];, you get the broken reference behavior even though you thought you were doing something that made sense.
    Thursday, February 9th, 2006
    1:22 pm
    [lithiana]

    so we just got our hands on a nice new Niagara server for evaluation:

    SunOS hydra.pmtpa.wmnet 5.10 Generic_118822-27 sun4v sparc SUNW,Sun-Fire-T200

    i’ve not had time to benchmark it yet, but first impressions are good... the ALOM management is useful (and a lot better integrated than Sun’s AMD stuff). it’s a bit more expensive than the equivalent AMD stuff, but hopefully if it’s fast enough we can save a bit of money on rack space and power (which we’re using an awful lot of right now) - one 1RU T1000 is rumoured to be as fast as two dual-CPU Opterons, as well as using a third of the power and half the space...

    Wednesday, February 8th, 2006
    2:22 am
    [brionv]
    MSIE strikes again
    So I'm trying to clean up the MSIE hacks stylesheet to get right-to-left languages on Wikipedia working properly with IE7 Beta 2. I end up with a couple style rules that I want to apply conditionally when dir="rtl" is present on the <html> element, and IE7 now supports CSS attribute selectors, right?

    Well... sort of.

    I try something like html[dir="rtl"] #column-one and have no luck; it just doesn't apply. After some further testing, I find that apparently the dir attribute is some kind of magic exception -- you can make selectors on random attributes you make up if you like, but you can't make them with dir!

    Here's a little test page I whipped up, and rendering in IE7 beta 2 vs Firefox 1.5:



    Sigh...

    Posted a report on MS's feedback newsgroup.
    Friday, January 6th, 2006
    4:45 pm
    [woofiegrrl]
    I'm having a problem with specifying a default for a parameter in a template.

    If I give the parameter in the template as: {{{Param|None}}}
    When the value is unspecified in a page I get: {{{Param|None}}} (printed plaintext)
    When the value IS specified in a page I get: {{{Param|None}}} (printed plaintext)

    If I give the parameter in the template as: {{{Param}}}
    When the value is unspecified in a page I get: {{{Param}}} (printed plaintext)
    When the value IS specified in a page I get: (the value printed plaintext)

    Basically it doesn't seem to recognize the parameter-with-default as being a parameter at all, it just prints it out plaintext regardless of whether it is specified or not. If I don't give a default, then without a value it prints the plaintext parameter, and with a value it prints the parameter properly.

    Does this make any sense? What might I be doing wrong? I have read the appropriate meta help carefully and as far as I can tell I'm specifying my default properly.

    Thanks in advance for the help!
    Sunday, November 27th, 2005
    9:01 pm
    [hashar]
    site status
    Edward Z. Yang (GPGKey:0x869C48DA) is working on an interface to show up wikipedia status: http://www.thewritingpot.com/wikistatus/

    It's currently alpha but you can probably test it and report bugs / wishes & issues to him. One day, maybe it will replace the old berlio status page at http://openfacts.berlios.de/index-en.phtml?title=Wikipedia_Status
    Tuesday, November 15th, 2005
    9:36 pm
    [hashar]
    squids mrtg is no more !
    Hello,

    A long time ago, gwicke set up mrtg to monitor squids usage. The system was available at http://wikimedia.org/stats/live/ .

    Someone else is generating new statistics and take care of Amsterdam, Seoul and Paris cluster as well. So you should update your bookmarks and have a look at the new system:

    http://noc.wikimedia.org/stats.php
    Tuesday, November 1st, 2005
    1:52 am
    [hashar]
    commons file server switched
    Yesterday, Brion announced a switch of the file server for http://commons.wikimedia.org/ . Should enhance download of pictures for all wikimedia projects (as long as the picture is on common).

    Commons is a wikimedia foundation project that aims to provide free multimedia materials. If you never browsed it, you should have a look at it RIGHT NOW.


    I even got a shell script that grab the picture of the day and set it as my wallpaper :)
    Thursday, September 22nd, 2005
    11:03 am
    [midom]
    horsepower!
    new application server boxes entering service:



    20 dual-opteron servers...

    but we still have to repair old db servers and obtain/deploy new ones.

    Current Mood: cheerful
    Monday, September 19th, 2005
    10:02 am
    [skagirlie]
    default search all?
    Hi.

    My library is going to be using a wiki for subject guides. We're using templates as side bar info thingies. Is there a way to set the search settings to default to searching templates as well, WITH OUT having to be logged in? We want our users to be able to find info in these templates with a minimal amount of effort.

    any help is appreciated!
    Friday, September 9th, 2005
    1:38 am
    [lithiana]
    well, the first three machines in the new Yahoo! cluster are now online. after some issues with installing the rest of them are resolved, we should have a nice shiny new cluster to do things with.
    Friday, August 26th, 2005
    9:38 pm
    [hashar]
    Wikimedia download portal updated
    I changed the download portal at http://download.wikimedia.org/ . Should be easier to find and download the dump.
    Thursday, August 25th, 2005
    4:47 pm
    [hashar]
    Portuguese translations updated
    Hello !

    Thanks to Get_It, we updated the Portuguese translation files on WikiMedia servers. The old one was Brazilian portuguese and got moved under PT_br.

    Every Portuguese speaker should now have a correctly translated interface on every WikiMedia sites.

    If you want to help with MediaWiki localisation, have a look at the guide: http://meta.wikimedia.org/wiki/MediaWiki_localisation
    Saturday, August 20th, 2005
    5:22 pm
    [dan_100]
    1.6 alpha
    The sites are now running MediaWiki 1.6 alpha, the CVS HEAD.

    This means new features will appear live on the sites much more quickly. On the other hand changes will have to be tested thoroughly, possibly on their own experimental branch.

    Users will see little new with this change for the moment, except a kind of 'edit conflict' page if a page you're editing gets deleted.

    More from Tim Starling: http://mail.wikipedia.org/pipermail/wikitech-l/2005-August/031103.html
    Monday, August 15th, 2005
    8:42 pm
    [lithiana]
    well, we've now had two of our nine Sun v20zs fail (in two months), the second one leaving mail offline for the last day.

    i shall think twice about recommending Sun x86 hardware in future ;)
    Wednesday, August 10th, 2005
    12:57 am
    [lithiana]
    i've been fiddling with mail.wikimedia.org a bit recently. while moving it to another machine, i thought i'd look at adding some anti-spam filtering, for those poor mailing list admins. i've written a brief thingy here describing how it works, if anyone's interested. seems quite effective so far...
[ << Previous 20 ]
About LiveJournal.com