chez  ·   jad   ·  tsurc   ·  cat   ·  tarquin  ·  cryptoboy Weird People    
journal.terryfroy.com
PhotosWritings
 
admin

post to journal
edit journal entry

archives

june 2004
july 2004
august 2004
september 2004
october 2004
december 2004
  january 2005
february 2005
may 2005
june 2005
november 2005
  january 2006
february 2006
april 2006
may 2006
july 2006
august 2006
september 2006
october 2006
november 2006
  february 2007
april 2007
may 2007
june 2007
july 2007
august 2007
september 2007
october 2007
november 2007
december 2007
  january 2008
march 2008
april 2008
july 2008
november 2008
december 2008
  march 2009
july 2009
august 2009
september 2009
october 2009
november 2009
december 2009
  january 2010
march 2010
may 2010
august 2010
november 2010
  march 2013

contact me

e-mail [pgp key]
homepage
icq

daily news

bbc radio 1
bbc news worldwide

fun stuff

ntk
fuckedcompany.com
bofh archives
the onion

internet oracles

google [usenet]

pc entertainment

c64 radio
project ay
world of spectrum
mame [unix] [wip]
id software
unreal tournament

network stuff

iana
6bone
rfc editor
arin whois
apnic whois
ripe whois

essential software

fedora core
courier mta
pureftpd
user mode linux

seo fun

uk tv abroad
live uk tv
website design lincolnshire
sticky labels

 
Saturday, March 27, 2010

Speeding up our little bit of the web...
(posted at 08:48PM GMT)

Recently, I have made quite a lot of noise about our new shared hosting platform because not only does it provide total security at the filesystem level, it also provides super-mega-speedy PHP interpreting and a dedicated MySQL box available over Gig-E interconnect.

Looking at the bigger picture, all we have accomplished so far is to reduce the total delivery time at the server - it doesn't actually get the content to the user as quickly as possible.

Fact: Manchester is approx. 9ms from London via our dedicated DWDM link; ergo, for every single HTTP request involved in a page, it adds 9ms to the RTT (round-trip time).

This isn't a big deal for a large download consisting of a single file as it involves a single HTTP request and 9ms doesn't really figure into the equation when it will take a few minutes to download that file.

When you are talking about small resources such as CSS and low-res images such as icons and page navigation elements, you can easily have up to 50-60 of those on the initial page and assuming everything else is optimal, that would easily add 0.3 seconds to the page load times which may already be up to 3 or 4 seconds depending on how well-structured the initial HTML is and how fast the webserver is ;-)

The solution usually is to build a CDN like Akamai, move the webservers physically closer to the users (i.e. in a London datacentre) or provide something that most hosting companies don't offer.

Like a HTTP/HTTPS accelerator/reverse-proxy.

Rather than have 50+ webservers all belonging to different customers sat in a London datacentre chewing up lots of power, we stick a couple of high-end 1U boxes with craploads of RAM and RAID-5 arrays with substantial write-back caches.

A cache is only able to speed up the serving of static content and can only 'pass through' requests for dynamic content, you might think there was extra latency caused for dynamic stuff but it isn't really the case as the majority of our UK traffic is now flowing through our London PoP so the latency between London and Manchester has merely shifted from Layer 3 (IP) to Layer 7 (Application).

Fact: Our London PoP is in Suite 1/G at Telecity Hex 8/9 and the London INternet eXchange Hex 8/9 PoP is in the suite next door ;-)

On one particular site we are trialling it with, the RTTs for static resources have dropped to microseconds rather than milliseconds.

More details will be made public when the service is ready for open beta to customers but it will definitely be free to customers on the existing shared hosting platform.

Tuesday, March 23, 2010

Back in the old town!
(posted at 10:35PM GMT)

Funny how quickly two and a half years can pass...

It felt right at the time to pull all our kit from Telecity Memaco House (http://journal.terryfroy.com/?year=2007&month=08) due to rising costs but since the network has grown somewhat out of control over the last six months, I thought that it would be beneficial to re-establish a PoP (point-of-presence) back in London.

While it technically went 'live' on Thursday, a lot of fine-tuning has been going on behind the scenes while it has been routing traffic; still trying to sort out ADSL backhaul but one of the suppliers is an absolute bloody nightmare to deal with on that score.

I'm still working on the new ADSL platform which supports IPv6 in a more stable fashion than the previous platform plus also does multi-link PPP which should bring in some interesting business opportunities when a certain LLU operator starts unbundling exchanges in Lincolnshire - customer can have two lines in two separate DSLAMs, bonded for performance 99.9% of the time but would still receive service in the event of a single line failing.

There is also one other service in the pipeline which should provide our Manchester-hosted customers with even lower RTTs for HTTP requests.

A couple of other skunkworks projects are also 'in progress' but I'll disclose a bit more detail on those if/when they ever get off the drawing board and into a real saleable product.

Saturday, March 13, 2010

Lazy fickwits...
(posted at 11:23PM GMT)

I have just finished moving a site written by a well-loved Lincolnshire-based web design company from one of their servers to our new shared webserver and quickly discovered that their poor example of a content management system is logging about two megabytes of data *per* dynamic page request to the associated error log for that site.

The number of disk writes per request is already staggering but when you also consider the in-process overhead of passing 2MB of data from PHP's stderr to the webserver process that actually writes the data to disk, it really doesn't surprise me that their webservers, which run hundreds of copies of this same code for a variety of different websites, perform like they have dog turds shoved in their Ethernet ports.

The error log is filled with this:

PHP Warning: mysql_fetch_array() expects parameter 1 to be resource, boolean given in /home/www.xxxxxxxxx.co.uk/public/xxxxxxx/xxxxxxx.php on line 249

Line 249 is this:

while(($row = mysql_fetch_array($SQLresult)) !== FALSE) {

... so I edited it to this:

while($row = mysql_fetch_array($SQLresult)) {

It still generates a single error on Line 249 per page request but doesn't cause a shitstorm of them like the previous code did; the change chops the total amount of errors per request down to a mere 11KB which is much much lower than 2MB.

The only reason I'm making such a loud noise about this is because the folks concerned already know about the amount of error-generating code present in the codebase but say that fixing it would take too long - as someone who knows next-to-nothing about their codebase, I managed to fix that single error in a matter of seconds and dropped the average response time to that request down from a second to a few milliseconds.

Thursday, March 11, 2010

Definitely faster....
(posted at 09:43PM GMT)

I am now down to the last hundred or so sites left to move across to our new shared webserver (see my January post on the subject).

There are currently just over 3,500 sites on the box; of which, just over 75% of them use PHP and over 90% of those use a MySQL database (which is provided via dedicated Gig-E interconnect to a box which just does MySQL).

Wanna see the load averages ?

[admin@fluffle ~]$ uptime
21:47:40 up 60 days, 12:41, 1 user, load average: 0.00, 0.00, 0.00

The box pushes an average of 6Mbit/s at peak times dropping to about 2-3Mbit/s during off-peak hours - yep, it is currently running half-load but still only has a load average of 0.00.

Compare that to a box running Apache/mod_php (not ours but managed on a customers' behalf) with a few hundred sites on it:

[xxxxx@xxxxxxx ~]$ uptime
21:43:41 up 245 days, 6:13, 1 user, load average: 0.54, 0.53, 0.95

... and a similar box with approx. same number of sites running Apache/mod_php/suPHP:

[xxxxx@xxxxxxx ~]$ uptime
21:52:00 up 23 days, 13:07, 1 user, load average: 0.77, 0.72, 0.55

On average, the two Apache boxes combined do less than 30% of the traffic of our shared webserver.

To be completely fair and to ensure that MySQL isn't unfairly blamed for the excessive load averages on these boxes, I'll throw the load averages from our dedicated MySQL box on the fire:

[admin@sentinel ~]$ uptime
21:52:59 up 44 days, 9:46, 1 user, load average: 0.00, 0.00, 0.00

The dedicated interconnect runs at about 10-15Mbit/s sustained at peak rates with it dropping to around 8-9Mbit/s during off peak hours; quite a few phpBB forums on our shared webserver and the search functions do get used quite heavily.

Based on the above data, it certainly looks like Spilsby Internet Solutions is 'technically leading' the way when it comes to high performance/low latency web hosting ;-)

Thursday, March 4, 2010

Non-CPU intensive spam filtering...
(posted at 11:52PM GMT)

'tis amazing what can be done when you mix IP-based blacklists, URL-based blacklists, SPF checks, DKIM policy checking and validation, greylisting, A/V scanning and some pretty vicious checks for RFC compliance into your border mailserver configuration.... and some tarpitting for the more persistent offenders who just won't accept several '550 Piss off' messages as a hint :-)

Without using any kind of anti-spam software that does Bayesian or Markovian analysis, I am seeing <1 spam every five days and considering the volume of legitimate mail I receive per day (usually 100+) that is pretty damn impressive.

I have just added Spamhaus' newly released Domain Blackhole List (DBL) into the mix and it is catching a few customer-addressed mails that SURBL and URIBL didn't catch, so that one addition is reducing the figure even further.

While I don't particularly trust IP-based blacklists due to the fact that you aren't classifying the message as spam, merely the source; my current attitude towards IP-based blacklists is that a source IP must be on a minimum of two independantly-run list (with condition that one *must* employ the use of honeytrap mailboxes) before messages from it will be blocked - better to let a few spams through rather than block legitimate mail from an erroneously-listed server.

URL-based blacklists are a different matter; if the message links to a spamvertised-site, it is, by definition, spam - so if it shows up on one of those lists, it will automatically be dropped on that basis.

The config 'recipe' will remain secret for now as it is the work of several months' careful tweaking and ordering of checks to ensure minimum CPU wastage and maximum efficiency.

 
slashdot

US Bans Noncompete Agreements For Nearly All Jobs

Generative AI Arrives In the Gene Editing World of CRISPR

Try Something New To Stop the Days Whizzing Past, Researchers Suggest

Oracle Is Moving Its World Headquarters To Nashville

Change Healthcare Finally Admits It Paid Ransomware Hackers

What Comes After OLED? Meet QDEL

The Ray-Ban Meta Smart Glasses Have Multimodel AI Now

HashiCorp Reportedly Being Acquired By IBM

Ex-Amazon Exec Claims She Was Asked To Ignore Copyright Law in Race To AI

Linux Can Finally Run Your Car's Safety Systems and Driver-Assistance Features

iPhone Sales Drop 19% in China

AI Is Poisoning Reddit To Promote Products and Game Google With 'Parasite SEO'

How GM Tricked Millions of Drivers Into Being Spied On

Apple Cuts Vision Pro Shipments As Demand Falls 'Sharply Beyond Expectations'

FTC To Vote On Noncompete Ban

the register

China's mega-telcos are spending billions on AI servers

Senate passes law forcing ByteDance to sell off TikTok – or face a US ban

US government reportedly ponders crimping China's use of RISC-V

White House tweaks HIPAA to shield medical files of those seeking reproductive care

Intel Foundry ticks another box in quest to fab mil-spec chips for US DoD

Using its own sums, AMD claims it's helping save Earth with Epyc server chiplets

Waymo robotaxi drives down wrong side of street after being alarmed by unicyclists

Banned Nvidia GPUs sneak into sanction-busting Chinese servers

Miles of optical fiber crafted aboard ISS marks manufacturing first

Seagate joins the HDD price hike party, blames AI for spike in demand

SpaceX workplace injury rates are rocketing

Miracle-WM tiling window manager for Mir hits 0.2.0

GM shared our driving data with insurers without consent, lawsuit claims

iPhone sales dive 19.1% in China as Huawei comeback hits Apple in the high end

Microsoft shrinks AI down to pocket size with Phi-3 Mini

Digital Realty wants to turn Irish datacenters into grid-stabilizing power jugglers

Microsoft really does not want Windows 11 running on ancient PCs

SAP cloud swells its topline, but profits slide

Mandiant: Orgs are detecting cybercriminals faster than ever

UnitedHealth admits IT security breach could 'cover substantial proportion of people in America'

Voyager 1 regains sanity after engineers patch around problematic memory

Leicester streetlights take ransomware attack personally, shine on 24/7

Silicon Valley roundabout has drivers in a spin

Don't rent out that container ship yet: CIOs and biz buyers view AI PCs with some caution

Over a million Neighbourhood Watch members exposed through web app bug

Meta comms chief handed six-year Russian prison sentence for 'justifying terrorism'

European Commission to suspend TikTok's new rewards program, open second probe

Misconfigured cloud server leaked clues of North Korean animation scam

Australia secures takedown order for terror videos, which Elon Musk wants to fight

Japan to draw up routes for roads dedicated to robot trucks

Old Windows print spooler bug is latest target of Russia's Fancy Bear gang

Protest group says Google has fired more staff over sit-ins opposing work for Israel

Tokyo wags finger at Google for blocking Yahoo Japan!from using ad tech

FBI and friends get two more years of warrantless FISA Section 702 snooping

Ex-CEO of 'unicorn' app startup HeadSpin heads to jail after BS'ing investors

Huawei wants to take homegrown HarmonyOS phone platform worldwide

Tesla slashes vehicle and self-driving-ish software prices as shares plummet

OpenAI CEO Sam Altman invests in solar power firm Exowatt to fuel AI datacenters

Tiny11 Builder trims Windows 11 fat with PowerShell script

Europol now latest cops to beg Big Tech to ditch E2EE

Fedora 40 is just around the corner with more spins and flavors than ever

Germany arrests trio accused of trying to smuggle naval military tech to China

Lawsuit accuses Grindr of illegally sharing users' HIV status

Gone in 35 seconds – the Cybertruck's misbehaving acceleration pedal

Watchdog tells Dutch govt: 'Do not use Facebook if there is uncertainty about privacy'

More than a third of enterprise datacenters expect to deploy liquid cooling by 2026

US House passes fresh TikTok ban proposal to Senate

Ex-Amazon exec claims she was asked to ignore copyright law in race to AI

Copilot auto-launch bug now takes flight in multiple Windows Insider channels

UK data watchdog questions how private Google's Privacy Sandbox is

 

Linux

Apache

PHP