Showing posts with label technology. Show all posts
Showing posts with label technology. Show all posts

Wednesday, June 8, 2011

Counting Bytes

Let me start by saying I never thought per byte billing was a good idea.  But it is what the industry is largely stuck with.  Why? Because we're too dumb to understand other methods.  Nevermind that we're too dumb to really understand bytes, but it is a concept we think we understand.  We don't.  Here's a a brief attempt to explain.

It seems simple to say you get charged a fixed rate for a chunk of downloads.  Say 10 cents for 1 gig of data.  Use 10 gig and pay $1.  Totally cake, right?  Well... did you mean gigaBITS or gigaBYTES?  A byte is 8 bits... so if you bought 10 gigabits you got 8 times less than 10 gigabytes.  Now... does gig mean 1,000,000,000 to you? or 1,024,000,000 to you?  You see a kilobyte is actually 1024 bytes... not a 1000.  Back in the day this difference was tiny and easily ignored.  Today we see the error maginified as kB moved to MB and then to GB and someday to Terrabytes. 

So immediately we find potential confusion in byte counting without even getting in to the technology.  Let's look under the hood...  Let's focus on download bytes as that is what most of us care about and pay for.  We go to YouTube and download 1 gigabyte of video for 10 cents.  So our usage should be 1 Gigabyte, right? Wrong... While it will be pretty close to 1 gig, we run in to problems with overhead.  You see the video you watched was "user data" or "payload".  It was packaged up by YouTube's servers and then sent to other network devices that added their own information in order to send it.  There are some control messages also be sent back and forth to verify all the data was received.  This extra chunk of data is considered "overhead" to you and I, but we couldn't get the funny kitten videos if we didn't have overhead.  So who pays for overhead? Me or the Provider?  Well usually consumers do.  So if I was just counting video size, I'll be a little low of the actual bytes sent.  How low? Well that's a really hard question since it depends on dozens of factors.  How many clips did I download? One big download will have less overhead than many small downloads.  How busy was the line between me and YouTube? If the line was busy, there is a chance a chunk of data got mangled or dropped and had to be re-sent (more overhead).  These factors go on and on making exactly how much overhead experienced impossible to predict.  So again, without looking very hard we've found confusing areas in byte counting.

Now consider malice.  As in, malicious software.  You may be running the cleanest PC on earth completely free of viruses and spyware, but thousands of other users are not.  Their PC's are frequently scanning zombies sending out megs of probes to random address ranges looking for other unprotected hosts.  This background noise on the Internet is constant and unstoppable.  The best we can do is filter out the worst and tell consumers to use various protection techniques.  But a Provider can not fully stop all the background noise.  These unsolicited bytes are sent to you and the Provider counts them against your bill.  The full amount of these will also vary widely on conditions.

Lastly, we have plain ol' boneheaded ignorance.  We don't know how much we're downloading.  We don't realize that CNN's homepage is a few hundred megabytes.  We don't know how big the pop-up ads are.  We don't know how big that video stream really was.  We don't know how big Window's Update really was.  There are million things communicating on consumer PCs these days (and Playstations and networked game consoles).  We are woefully unaware of these transactions.

Byte counting is terrible.  It is always going to be open to very wide interpretation.  Your Provider is always going to seem a bit high and what you're really downloading is (for now) always going to be largely a mystery unless you're a network monitoring professional with a packet analyzer connected on your outgoing link... which is a challenge even for hardcore geeks to get good analysis on DSL or Cable modem lines.

So far this has all been pretty simple concepts.  From here it gets much more complex, but no less confusing and significant.  There are a million valid ways your Provider could be counting bytes.  It will barely make sense to them let alone the customer.  The only thing they have going for them is that whatever system they use will likely be consistent across all consumers.  That is if it 10% wrong, it will be 10% wrong all the time.  As consumers we have to live with it.  Don't ask your Provider to tell you how they do it.  It is unlikely you'll get a coherent answer.  Even if you were an expert in billing systems it would likely take you weeks of research to plot exactly how the byte counting is done from one Provider to the next.

Counting bytes is what we're stuck with for the short term.  But don't get too hung up about accuracy.  The question is too subjective.  I think the only thing we can tell our service providers is "we want it to cost less".  We don't care how.  Just make it cost less... OR be more valuable to us.  But please... lose the byte counting.  It has become meaningless.

Wednesday, April 21, 2010

Sci-Fi Vision

I had one of those artistic leaps that you have to jot down as a potential story plot/concept.

A friend of mine linked a news article about a man losing his memory after hitting his head.  It resulted in what they think may be one of the most extreme cases of memory loss ever recorded.  He lost everything before the bump.  He has been naturally struggling not just with remembering his family and friends, but literally who he is.

This sparked the question of "what would you do with a clean slate?" Which I didn't find all that interesting, but someone wrote back that they would check their Facebook site for all their contacts and things they wrote about or posted.  To them Facebook would provide a wide commentary on "who they were."  It was this thought that struck my brain.  That now, here in good old 2010, we may be seeing the very first fledgling digital personality archive... a concept that will grow over the next 100 years to ever more immersive and "real" snippets of our experiences in life. 

I've read sci-fi stories before that talked about you could "download" your consciousness to a new clone and thus be virtually immortal.  But I never quite connected that social networking sites (taken forward a few decades) might actually be the means by which we accomplish this.  We may publish ourselves in modified form.  Perhaps we can even re-program ourselves to kick those bad habits we don't like.  Our Super Facebook identities will be the true expression of who we wanted to be.  Our "site" would be filled with a life-time's worth of content and commentary.  Our families and ancestors could review our whole lives.  We won't need a programmer to "code" our personality... we'll do it ourselves.  While downloading to a different biological body my be a much farther ways off... it isn't too hard for me to imagine a rough AI of myself built around 50 years of blogs/facebook entries.  My great-great grandkids might get a chance to talk to me and ask my avatar what I liked, who I was in love with, what tasks I found most challenging at work, or any other thing I may a felt inclined to post or write about--assuming I have not downloaded to a clone or just uploaded to the Internet... 

Wednesday, March 11, 2009

On Kindle, Twitters, and Publishing

Some of you may recall that Oprah did a show back in October featuring the Kindle device from Amazon.  I must admit that I missed it.  For those of you that don't watch Oprah or know what the Kindle is (pictured), the Kindle is an electronic widget that downloads books on to a high contrast large display screen.  It's wireless (uses the Sprint network as a matter of fact) and can store hundreds of titles--each purchased off Amazon's website for a fee from a few dollars to $20 USD.

Why is this blog-worthy?  Well it is a demonstration of a new publishing era.  If you fancy yourself an amateur author (all you NaNoWriMo's) then all you need to do nowadays is convert your masterpiece to a PDF file and post it for sale.  Amazon will even take your book (for a piece of the profit).  Other sites exist to post your books like...

http://www.smashwords.com

http://www.fictionwise.com

...and I'm sure many others.  This is pretty new business and probably a good time to be looking at it.

Publishing in this way has a slim fraction of the overhead costs of a paper printing.  Authors can get a much larger portion of the purchase price.  Digital media will never completely replace the book, but it is a nice way to dabble and take some risks with new authors.

There are other devices as well.  The Kindle2 is probably the coolest, but it only works in North America--due to that wireless dependence on Sprint.  But other versions of readers sync to your computer and therefore can store all the same content for travel--you just won't be able to download from the airport.  Unless, that is, you travel with a laptop.  Clunky, but cheaper.  The Kindle2 comes in at $359 USD.  That's a chunk for most folks--even in America.  But readers will likely come down.  Certainly if readership picks up (and thus purchase revenue increases) you'll see more subsidized devices--where Amazon give the device away cheaper to facilitate more eBook sales.

Stephen Fry is a UK comedian and actor.  While you might remember him from the blockbuster film "V is for Vendetta" (also starring Natalie Portman), he is more recently famous for being the biggest user of Twitter on the planet.  Stephen has over 250,000 followers on the trendy new application.  Twitter lets you type out a short 160 character message that will be broadcast to everyone that choses to follow you.  These messages are usually delivered to your mobile phone via SMS (thus the 160 character limit).  Why is Stephen important to this story?  Well Stephen has recently purchased a Kindle while on holiday in America.  He's broadcasting his positive experiences with the device to all 250,000 of his followers.  While this is only a small fraction of Oprah's viewership, it is a more mobile device oriented crowd.  There is a powerful surge of marketing behind the Kindle now.  Digital publishing is going to increase in the next few years.  Will it hold out as a lasting media? Who knows?  But certainly the short-term outlook is all positive growth.

Monday, February 23, 2009

Adult Content Warning

So I decided to share a piece of cyberpunky fiction with some friends who ask for it.  I wrote the whole thing on my spare time, but kept the file on my work laptop for convenience.  I haven't opened the file since before our Holiday break in December nor did I open it today.  I simply attached it and forwarded it.  About a second later I got this automated message...

Your email message RE: Cyberpunk? to "Ben Wilson" has been delivered.

However it was identified by our content filtering processes as containing language that may be considered inappropriate for business-like communication. It is possible that the recipient(s) may not have received it if they run similar filtering processes. You may want to confirm with the recipient(s) that they have received the email correctly.

A copy of this email has been redirected to an administration queue which is randomly scanned for inappropriate mail and objectionable material.

Email that fails to comply with ******* corporate policy may result in disciplinary action being taken.

The ******* corporate email policy can be found at

The content which caused this to happen was

List:Reputation Protection Keywords

Found the expression "porn*" 2 times, at 2 points each, for an expression score of 4 points.

Found the expression "bitch*" 1 times, at 3 points each, for an expression score of 3 points.

Found the expression "*fuck*" 2 times, at 1 points each, for an expression score of 2 points.

Found the expression "Fuck" 4 times, at 9 points each, for an expression score of 36 points.

Found the expression "nasty" 1 times, at 1 points each, for an expression score of 1 points.

Found the expression "rape" 1 times, at 2 points each, for an expression score of 2 points.

Found the expression "sex" 2 times, at 2 points each, for an expression score of 4 points.

Found the expression "shit" 2 times, at 1 points each, for an expression score of 2 points.

Found the expression "naked" 2 times, at 1 points each, for an expression score of 2 points.

=============================================================

Total Message Score: 56 points.

Woo! 56 points on the naughty system!

Of course, I didn't know we had a naughty word scanner on the network... I suppose that was money well spent to find all the folks who might be sending naughty words.  I mean if you sent an offensive work message, I bet the recepient would complain to their supervisor.  Then they could lookup the naughtiness point value... maybe they have a threshold where 50 points is a misdemeanor and 100 points is a felony.

Just who sat down and coded which words were naughtiest? How exactly did they accomplish this on their own network without tripping their own alarms...  What about the autoresponder itself... If I fowarded this to my boss, would it get in trouble too?

Anyway folks... The data gods are watching.  A strange irony to be detected in a cyberpunk story...

 

Adult Content Warning

So I decided to share a piece of cyberpunky fiction with some friends who ask for it.  I wrote the whole thing on my spare time, but kept the file on my work laptop for convenience.  I haven't opened the file since before our Holiday break in December nor did I open it today.  I simply attached it and forwarded it.  About a second later I got this automated message...

Your email message RE: Cyberpunk? to "Ben Wilson" has been delivered.

However it was identified by our content filtering processes as containing language that may be considered inappropriate for business-like communication. It is possible that the recipient(s) may not have received it if they run similar filtering processes. You may want to confirm with the recipient(s) that they have received the email correctly.

A copy of this email has been redirected to an administration queue which is randomly scanned for inappropriate mail and objectionable material.

Email that fails to comply with ******* corporate policy may result in disciplinary action being taken.

The ******* corporate email policy can be found at

The content which caused this to happen was

List:Reputation Protection Keywords

Found the expression "porn*" 2 times, at 2 points each, for an expression score of 4 points.

Found the expression "bitch*" 1 times, at 3 points each, for an expression score of 3 points.

Found the expression "*fuck*" 2 times, at 1 points each, for an expression score of 2 points.

Found the expression "Fuck" 4 times, at 9 points each, for an expression score of 36 points.

Found the expression "nasty" 1 times, at 1 points each, for an expression score of 1 points.

Found the expression "rape" 1 times, at 2 points each, for an expression score of 2 points.

Found the expression "sex" 2 times, at 2 points each, for an expression score of 4 points.

Found the expression "shit" 2 times, at 1 points each, for an expression score of 2 points.

Found the expression "naked" 2 times, at 1 points each, for an expression score of 2 points.

=============================================================

Total Message Score: 56 points.

Woo! 56 points on the naughty system!

Of course, I didn't know we had a naughty word scanner on the network... I suppose that was money well spent to find all the folks who might be sending naughty words.  I mean if you sent an offensive work message, I bet the recepient would complain to their supervisor.  Then they could lookup the naughtiness point value... maybe they have a threshold where 50 points is a misdemeanor and 100 points is a felony.

Just who sat down and coded which words were naughtiest? How exactly did they accomplish this on their own network without tripping their own alarms...  What about the autoresponder itself... If I fowarded this to my boss, would it get in trouble too?

Anyway folks... The data gods are watching.  A strange irony to be detected in a cyberpunk story...

 

Thursday, April 17, 2008

Free Broadband and File Sharing

This issue comes up a lot in my line of work.  95% of the folks that probably read this don't care about it--you can skip it.  If you are in the 5% that know techy talk and care, you can follow my ranting.

Here's the deal folks... Network equipment and leased circuits that carry your Internet traffic are FREAKIN' EXPENSIVE.  We're talking millions of dollars for a small network (to serve say 25,000 subscribers) in both one-time equipment costs and ongoing (monthly) circuit and maintenance cost.  To say nothing of Customer Care, Sales, Marketing, etc.  I know it totally feels like it just falls out of the sky, but there are literally thousands of miles of infrastructure and probably several thousand people involved in every bit you send.  It ain't free.

The only way anyone can make money in this game is to over sell.  That is you build a network capable of 100 meg of total service and then sell 5,000 one meg connections to your subscribers.  This is entirely legitimate as most users *never* tap one meg download speeds and if they do, they don't do it very long so the sharing works out.  5,000 customer get cheap reliable Internet service and they just have to agree to share it a little.  The service provider or carrier (that's me) has to monitor the total usage and upgrade when financially viable to do so.

Lately, more and more press has be coming up criticizing carriers for limiting heavy users on their networks.  Remember that in order for the cheap bandwidth model to work it has to be shared.  When a single user consumes 20 meg of the 100 meg aggregate, the other 4,999 subscribers are forced in to sharing the rest.  Everyone's performance goes down.  The responsible provider (that's me again) has to protect the bulk of its users (you) by limited what any one consumer can monopolize.

Yes, your equipment can peak out at 10 meg a second.  But that is NOT guaranteed to you 24x7.  If it were *guaranteed* then my 100 meg network would have to be a 50,000 meg network to physically guarantee that my 5,000 subscribers would *always* have their 10 meg peek available.  This network will, in fact, cost 1,000 times as much as the 100 meg network.  Soooo... if you were paying $40 a month for service before, you are going to have to pay $40,000 a month for it.

What? You can't afford $40,000 for high speed guaranteed porn, pirated movies, pirated songs and pirated software?  What? You don't do any of those things?

Of course not.  Most consumers don't do those things and because of this, they have reduced bandwidth needs.  The ability to burst occasionally is all you need.  It is only the 10% that "demand" their traffic be treated "fairly".  By fairly, they mean exclusive of anyone else's rights to use or the carrier to make any profit on it at all.  They are also screaming for this free and fair access so they can pirate their weaselly guts out.

This hypocrisy knows no bounds.  To pound the table about how it isn't "right" to restrict their bandwidth while they break the law, steal anything they can download and degrade network peformance for all.  Now even in that 10% there are relatively few who are the grossest offenders.  These individual literally down load terabytes of stolen software and media.  They do it for no reason other than bragging rights about their access to anything.  Many of these same individuals also download the vilest porn you can't imagine--often this stuff is deplorably illegal.  The good news here is that this type of traffic is easy to identify on the network.  As a carrier I can categorize it and shunt it off to rate-limited links that allow it, but keep it off the 100 meg shared by everyone (this isn't exactly how it works, but it is easier to explain this way).

Now I am a fair minded guy.  I do not care what weirdness you are in to.  The Internet has a place for you.  Go for it.  Do whatever you can do.  But you can not expect the rest of society to pay for your terabyte downloading habits.  This traffic needs to be controlled and should be controlled.  I do not even suggest that it should be banned--only restricted so that it can not overwhelm the rest of the network.  That is what carriers are doing today and they should.  The individuals that are banding together to some how legislate against the carriers are arguing that their right to download illegal media and software supersedes your right to affordable Internet access.  If you force a carrier to allow all traffic "unlimited" then that carrier is going to have upgrade and pass on the cost to you the 90% user who never (or rarely) taps their full line and never (or rarely) indulges in file sharing.

Go download what ever you conscious allows, but don't be dumb and make your carrier charge you more because of a small number of abusers.

Free Broadband and File Sharing

This issue comes up a lot in my line of work.  95% of the folks that probably read this don't care about it--you can skip it.  If you are in the 5% that know techy talk and care, you can follow my ranting.

Here's the deal folks... Network equipment and leased circuits that carry your Internet traffic are FREAKIN' EXPENSIVE.  We're talking millions of dollars for a small network (to serve say 25,000 subscribers) in both one-time equipment costs and ongoing (monthly) circuit and maintenance cost.  To say nothing of Customer Care, Sales, Marketing, etc.  I know it totally feels like it just falls out of the sky, but there are literally thousands of miles of infrastructure and probably several thousand people involved in every bit you send.  It ain't free.

The only way anyone can make money in this game is to over sell.  That is you build a network capable of 100 meg of total service and then sell 5,000 one meg connections to your subscribers.  This is entirely legitimate as most users *never* tap one meg download speeds and if they do, they don't do it very long so the sharing works out.  5,000 customer get cheap reliable Internet service and they just have to agree to share it a little.  The service provider or carrier (that's me) has to monitor the total usage and upgrade when financially viable to do so.

Lately, more and more press has be coming up criticizing carriers for limiting heavy users on their networks.  Remember that in order for the cheap bandwidth model to work it has to be shared.  When a single user consumes 20 meg of the 100 meg aggregate, the other 4,999 subscribers are forced in to sharing the rest.  Everyone's performance goes down.  The responsible provider (that's me again) has to protect the bulk of its users (you) by limited what any one consumer can monopolize.

Yes, your equipment can peak out at 10 meg a second.  But that is NOT guaranteed to you 24x7.  If it were *guaranteed* then my 100 meg network would have to be a 50,000 meg network to physically guarantee that my 5,000 subscribers would *always* have their 10 meg peek available.  This network will, in fact, cost 1,000 times as much as the 100 meg network.  Soooo... if you were paying $40 a month for service before, you are going to have to pay $40,000 a month for it.

What? You can't afford $40,000 for high speed guaranteed porn, pirated movies, pirated songs and pirated software?  What? You don't do any of those things?

Of course not.  Most consumers don't do those things and because of this, they have reduced bandwidth needs.  The ability to burst occasionally is all you need.  It is only the 10% that "demand" their traffic be treated "fairly".  By fairly, they mean exclusive of anyone else's rights to use or the carrier to make any profit on it at all.  They are also screaming for this free and fair access so they can pirate their weaselly guts out.

This hypocrisy knows no bounds.  To pound the table about how it isn't "right" to restrict their bandwidth while they break the law, steal anything they can download and degrade network peformance for all.  Now even in that 10% there are relatively few who are the grossest offenders.  These individual literally down load terabytes of stolen software and media.  They do it for no reason other than bragging rights about their access to anything.  Many of these same individuals also download the vilest porn you can't imagine--often this stuff is deplorably illegal.  The good news here is that this type of traffic is easy to identify on the network.  As a carrier I can categorize it and shunt it off to rate-limited links that allow it, but keep it off the 100 meg shared by everyone (this isn't exactly how it works, but it is easier to explain this way).

Now I am a fair minded guy.  I do not care what weirdness you are in to.  The Internet has a place for you.  Go for it.  Do whatever you can do.  But you can not expect the rest of society to pay for your terabyte downloading habits.  This traffic needs to be controlled and should be controlled.  I do not even suggest that it should be banned--only restricted so that it can not overwhelm the rest of the network.  That is what carriers are doing today and they should.  The individuals that are banding together to some how legislate against the carriers are arguing that their right to download illegal media and software supersedes your right to affordable Internet access.  If you force a carrier to allow all traffic "unlimited" then that carrier is going to have upgrade and pass on the cost to you the 90% user who never (or rarely) taps their full line and never (or rarely) indulges in file sharing.

Go download what ever you conscious allows, but don't be dumb and make your carrier charge you more because of a small number of abusers.

Wednesday, June 22, 2005

The politics of Celluar Technology

Since I'm heading out of town today to attend a conference on the future of data on CDMA networks, I thought I'd throw down some explanations of radio technology in layman terms.



When it comes to cell phone technology it boils down to this... The US is Red and the rest of the world is Blue.  Phones for the Red networks don't work on Blue networks and vice versa.  The Red and Blue networks both offer two basic services; Voice and Data. 



Voice is of course the original reason for Telephone Companies in the first place.  In the 90's we experience this little phenom called "the Internet" and suddenly Telephone Companies were suddenly interested in Data.  We all fondly remember the days of modems and dial-up.  Some of us are still living that dream... *cough* Then came digital access and then some fool decided that even wired digital access wasn't convenient enough and Telephone Companies delivered "wireless."



So back to Red and Blue networks... Wireless was a "Big Deal(tm)".  It was the greatest growth business for the Telephone Companies in a 100 years.  At the time it happened, Blue networks were established and Red networks were "experimental."  Europe went Blue and the US went Red.  Red worked better, but Blue was cheaper.  Countries often chose the cheaper techology.  Thus the world went Blue.  Just to clue you in, Red networks are CDMA and Blue networks are GSM.  Means a lot to you I'm sure.  Now this data thing... Blue networks did data right from the start--only no one cared at the time.  Red networks delivered data right when it mattered and it worked better than Blue.  However, since Blue had been technically doing data longer, Blue carriers had more kinks worked out.  You could use Data on anyones  Blue network.  This is called Data Roaming; much like Voice Roaming.  Now here's where the Red networks fell on their sword.  They didn't make their data work on other Red networks.  We're working on it.



The problem is now that Red is so popular (in the US and a few other places), there are a lot of Red carriers.  We all have to agree on changes if we all want to make money off you (the consumer).  All phone companies are big with too much money.  That makes them believe they are important.  Important people tend to want to dictate more than cooperate.  So that's where I'm going this week.  The Red network "lets all cooperate" shindig.  A 2 day festival of every CDMA carrier trying to be compatible while at the same time angling for a technical advantage over the other carriers.  The corporate expense budgets are going to flow...



Also worth mentioning in closing is the "Holy Grail" of this story.  Getting Red data users to work on Blue networks.  Technically we don't care about Blue subscribers... being a Red network and all.