Today a consortium of hardware and software vendors announced that they will be funding a non-profit organization that will design and offer monthly security challenges similar to the one that recently discovered a major problem in OS X. Anthony Lawrence, the newly appointed Director of Operations for this organization, explained that the motives of the funding companies aren't entirely altruistic:
Actually, they all realize that there are mutual benefits. So much software exists across multiple platforms that an exploit on one platform probably exists on another. Even if it doesn't, the concept of the exploit may point out danger points in other operating systems.
We hope that there may be other benefits. Because we are offering large cash prizes for demonstrated exploits, our hope us that at least some black hat hackers might be persuaded to augment their income by winning these contests rather than using their knowledge for illegal purposes. Even if that doesn't happen, many underground, "zero day" exploits will surely be brought to light by these challenges.
The challenges will be held monthly and will have prizes running from $5,000 to $50,000.00. All manner of hardware and operating systems will be included, from iPhones and Blackberries right up through supercomputers. Precise rules and the prizes offered will be published a month ahead and each contest will run until the prize is won, so Mr. Lawrence expects that there may be multiple contests running concurrently at one point. The contests will be similar to other contests where the prize for a "hands off" hack is higher than those that involve user involvement.
There's another slant to these contests: the "Duh Awards" for security lapses that should not have happened. These are tongue-in-cheek awards to companies and individuals who make egregious security slips or repeat the same mistake again. Lawrence explained that these are supposed to be in a spirit of fun ("We all make dumb mistakes sometimes", he noted) but also hopefully will provide extra incentive to be careful in code.
The first contest is expected to be announced in July of this year. Lawrence explained that because of the number of companies involved and their generous commitment to improving security, the available prize pool exceeds $100,000 per month.. "That should be enough to attract plenty of interest", he opined.
Yes, it is April first. But wouldn't this actually be a wonderful idea?
I think this is a very clever idea: JackPot Rewards is a discount shopping club ($3 weekly) that blends on-line discounts with a weekly lottery.
I took a look at this and I thought the stores and prices offered were reasonable - probably NOT worth $3 a week by itself (though the "Deal of the Day" stuff has been very attractive), but then there's that lottery aspect: we spend $2-$3 a week on lottery tickets now, and often forget to buy them, or worse forget to check them. Yes, of course it's a waste of money given the odds, but on the other hand..
Well, you get the point. I think a lot of people might just go for this idea, and I suspect that this isn't the end of it: we'll be seeing more of this kind of thing.
Only a few hundred people were members when the first lottery ran: that was pretty good odds. Of course there will be thousands and thousands by now, but it still might be better odds than State Lotteries.. and of course if you just take the free trial and cancel before they charge you, well, you invested nothing at all. Those odds are even better..
I don't know if we'll keep it (probably not), but we'll try it out for a month and see how we feel.. I left that up to my wife, Professional Shopper and Certified Bargain Hunter - if she finds it to be worth the charge, it really is. If not, she'll chop it off like a piece of gristle on the chicken she's cutting up for lunch - no doubt about that.
My point here is not to advertise this site, but simply to point out the cleverness of the idea. I showed this to a customer who is starting to get into Internet retailing - he had the same reaction I did: "Oooh, I wish *I* had thought of that!".
However, any good idea is bound to get over-used. If consumers do find this attractive (real consumers, like my wife, not dilettantes like me), clones will spring up and soon you won't be able to sell anything on line unless you also offer a lottery. Note: lotteries tend to get more popular in tough economic times, so our present monetary troubles could add to this if it does become a trend.
Oh, one more thing: I can almost guarantee that signing up for this will increase your spam level. Use a disposable email address if you are tempted. Oh, and of course you can also take advantage of the "no purchase necessary" clauses that all U.S. lotteries require - though if postage keeps going up that isn't exactly a free ride either.
If you install the Xcode Tools from your OS X CD, you get a lot of free development tools, and you also get the "setfile" and "getfileinfo" utilities. These can set or display a number of HFS+ file attributes that you may not have even known existed. Some of these seem to be either unimplemented or are perhaps for use only by certain programs, but two (visibility and locking) are useful for our use.
Locking
You can do this from the "Get Info" dialog also, but the command line method is "setfile -a L filename". After doing that, a "getfileinfo" will show a capital "L" in its listing:
Note that if a file is locked, you can't set this or any other attribute until you unlock it ("setfile -a l filename"). Even if you have set V, the file remains visible to "ls" in Terminal and will be visible even in Finder unless AppleShowAllFiles is "No" in your Library/Preferences/com.apple.finder.plist (that is the default, but you may have changed it). If you do need to change this back, you can use the Property List Editor from /Developer/Applications/Utilities, or just do this in Terminal:
defaults write com.apple.finder AppleShowAllFiles No
Or just vi the plist file directly - it's plain text.
You'd need to relaunch Finder (Apple Menu,Force Quit, choose Finder and click "Relaunch") for that change to take effect.
A chirpily enthusiastic blurb on the morning news convinced me to go take a look at Hulu, which in turn cheerfully promises:
Watch your favorites. Anytime. For free.
Hulu offers U.S. consumers a vast selection of premium video content, on demand, free and ad-supported: full episodes of TV shows, both current and classic, full-length movies, thousands of clips, and much more.
Except they don't. They offer a few episodes of popular shows, a few clips from new movies, and not much else. I won't be going back..
If they really had everything - and I mean EVERYTHING, because disk space is dirt cheap - this would be something worth paying for. Of course I want the same thing from my FIOS "on demand" service - I don't want to choose from what they want to offer this month - I want to watch what I want to watch when I want to watch it, on TV or at my computer.
Speaking of that, when are the content providers like Comcast and Verizon going to attach the Internet to my TV and give me a bluetooth keyboard and mouse to control it all with? I WANT the Internet in a PIP window so that I can look up things while watching TV and ideally I'd like to send my own computer's screen to it. Tell me how hard this would be for Verizon or Comcast: everything goes through the same router already: all it needs is an open VNC connection to my machine and the aforementioned disk farm.. there's absolutely NOTHING technically difficult here!
Do that, and I'll happily pay for it, either on an "as viewed" basis or by some "content bundle" pricing. But if they want me to willingly part with more money, they need to give me EVERYTHING..
It turns out that it's actually a QuickTime flaw and more specifically a QuickTime Java flaw.. or so they seem to be saying now.
There's been a lot of confusion, with some blogs mixing in the two year old Mac Mini challenge and confusing that with this. That challenge involved giving the attacker a user account to work from; this one was done indirectly through a web page.
Of course some Windows folks are cheering wildly, but that's silly: most Mac fans are happy the bug was found and will be fixed. I wish they'd do this kind of thing more often: Apple, Microsoft, and everybody else with an interest in improving our Internet world should be ponying up prize money so that this kind of testing could be done every month. That could only improve life for everyone, right?
The first few times I saw this, I thought I was just mistaken.. but it's happened so many times now I know it's not me, this is really happening: older Firefox pages reappear if I Alt-TAB away and then return to Firefox.
It's Gmail where I can most easily notice this, so it MAY have something to do with Ajax, but I THINK I've seen it on other pages too (confirmed: I just saw it on another page). It's pretty weird: let's say I've opened up in the morning to twenty unread Gmail messages. I read and delete 15 of them, then Alt-TAB off to do something else.. but when I come back, the messages I read or deleted show up as though I had never touched them. They really aren't: a "refresh" shows the correct view instantly, but it's quite confusing.
I know, you probably think I'm nuts - or have two Gmail windows open. It's definitely not the latter, and I don't think that taking my wife's appraisal of my mental state is valid. Really: this happens.
It MAY have something to do with Parallels: it seems like it happens more often if I've Alt-TABbed there.. but it's not repeatable - I've not found anyway to demonstrate this consistently.
Anybody else ever seen this? It might have something to do with running the Firefox 3 Beta too - I'm not sure if this started just after loading that..
Excuse me while I do my little "I told you so" dance.
If you'll follow that first link, you'll also learn that fourteen percent represented twenty five percent of the dollar volume - yes, twenty five percent, all for Apple. The snowball is growing..
Have you ever been in an Apple store? There are more and more of them popping up, so there is probably one near you. Just go take a peek: every time I've been to one it has been packed with people.. of course a lot of that is iPods and the like, but the 'puters are selling too.
And why? Because people are fed up with Microsoft and they are afraid of Linux. Because people who aren't afraid of Linux realize they can have both on one very nice machine. Because even places like PC Magazine are recognizing the superiority of Leopard over anything else out there.
Wait.. I have to do that little dance again.. sorry, it will just take a minute.
Apple is going to continue to pull market share from Microsoft (and so is Linux). Why? Because they make a better product. A better OS, yes, but also better hardware to run it on. Sexier hardware, better engineered hardware. No doubt the Windows fanboys hate that, but it's true: Apple innovates, and everyone else copies.
One more dance? I promise no giggling during this one..
A recent Information Week article doesn't even question the issue. They ask Which Apple Should You Pick? and say:
Apple's product lineup is clear, logical, and targeted at distinct types of users and uses. The entire product line is one of the strongest Apple's ever had. Without hyperbole, it may be the best overall line of computers anyone has had, ever -- there's not a bad Apple among the bunch, and some are truly superb.
Seriously now: you don't own a Mac yet? What's stopping you?
I'm beginning to really hate the old SCO systems that are left out there. However, when they call with a problem, I have to help, right? Well of course I do. But I can't help cringing sometimes..
This call was from a Boston client who honestly is trying to get off this creaking old boat. They can't seem to find off the shelf software that will meet their needs, but they have hired some company to write them a new app. Unfortunately it will be Windows, but that is what it is, right? Anyway, while the app development proceeds, the old SCO has to keep running. Early on a Friday morning, it stopped.
Well, not exactly. It stopped letting people log in. People who were already logged in could work, but recieved a message described to me as "something like 'system database not allocated'"
Well, OK - normally I'd complain and say "Please give me the EXACT message you saw", but for this I knew it had to be something in the TCB (Trusted Computing Base) being messed up. SCO has built in tools to examine and even fix up minor problems, so I felt I might be able to lead someone through fixing it, but on the other hand this stuff can sometimes get nasty so I wasn't sure. I had another problem too: I had to be somewhere else in fifteen minutes and I wouldn't be able to talk anyone through anything until I was done with that.
"How many people are still logged in?", I asked.
"Four"
Well, heck, that's not so bad. It's Friday, a slow day for them as it is for most businesses, and they only usually have about twelve or so people working on that system anyway. I asked if they could limp along for a few hours. Yes, they could.. but please hurry.
So I went off to my meeting, but couldn't help thinking about how I'd approach this problem. Most likely, I could just have them run "integrity" to get a list of damaged files and restore them from backup. Most likely..
That's assuming the backup is good, of course. What if the backup has messed up files or the problem is really somewhere else? I can look at a tcb file and know what I'm looking at, but it would just be gobbledy-gook to the person at their end, so I'd have to get them to print things and fax them to me.. yuck. I shortly convinced myself that it was better to go in to the job.
After finishing up my other business, I called and explained that. I said that I probably could fix this over the phone, but I was a little hesitant because it could get nasty, and I'd rather just come in. The people at the other end immediately agreed: they really didn't want to be led through anything anyway. By the way: they were now down to two people working because the other two had "accidentally" logged out.. no, I don't know how you "accidentally" log out either.
An hour later I was there. I ran "integrity: and it pointed to /etc/auth/system/default as the problem. I looked at it and found it zero length.. how could that happen? That file should look something like this:
I've never seen a system glitch just zero that, so I was suspicious.
"Who's got root?"
It was a short list.
"I think somebody mean to clear something else and somehow cleared this.. or.."
The "or" would be file system corruption. Possible, but no other indicationss of that. Nothing in "syslog" or "messages" indicating any other problems. I restored the file from backup and people immdeiately could log in. I watched log files for any other problem and ran "integrity" again. No problems..
I hung out a while checking this and that.. nothing seemed odd, so I really think someone had to have done this by mistake.. I have no idea what they thought they were doing.. it couldn't have been sabotage because it was too limited.. surely anyone wanting to cause damage would have done more.. well, I would think so anyway.
So that's where I left it. I'll check in on it a few times over the next week, but I really don't think there is anything wrong.. just an "accident".
Oh, I do get grumpy. You'll hear this on the radio and TV: "Just go to www.foobar.com backslash special offer!".. and, curmudgeon that I am, I start yelling "It's not a backslash, you bleeping idiot!"
That's embarassing. People stare at me - especially since they, deluded by Windows and reinforced by constant repetition in media, likely are quite sure that "/" is indeed a backslash - duh!
I really need to learn to just keep my mouth shut. If conveniently near an Internet connected computer I could quickly show them the error of their belief, but even then I've seen people shrug their shoulders and insist that "\" is a slash.. if Mighty Microsoft says so, it must be true, right? But, but, I splutter, Microsoft says no such thing. For example I submit http://support.microsoft.com/kb/870839 which plainly explains a bug in Microsoft Excel thusly:
When you open a Microsoft Excel for Windows workbook in Microsoft Excel 2004 for Mac, the yen character may appear as a backslash (\).
See? Even Microsoft knows what a backslash is.
It could be worse. If they say "Log in to www.foobar.com backslash special offer!" I turn purple. Of course it's always possible that you really will have to log in that site, but I doubt it. No doubt the people that write this junk think it's cool computer talk, but it of course it really isn't: it's dumb, I don't know beans about computers talk. And again it causes people to look at me like I'm the crazy one.. well, ok, maybe I was shouting a little..
Then there's the security stuff. Last night I watched an old NCIS episode.. just happened to flip into it as the boss told some young lass that she needed to break into a Pentagon computer. I'm not sure what the point of all this was, but soon that woman and another man were hacking away at their keyboards and the screen was full of hexadecimal dumps interspersed with blinking red "Access Denied!" lines. Oh yeah, I'm sure that's how the Pentagon does it - plain old passwords. And I'm sure they'd let you keep guessing as long as you like, too. Those attempts wouldn't attract any attention at all.. yeah, right.
Oh, TV can be so annoying.. I don't know why I bother yelling at it when no one is here to "shush" me.
Finally, I don't like blurred out logos, expecially when I can tell from the keyboard or the screen that yes, it is a Mac. What's the point of that blurriness over the Apple logo on the back of the cover? You tried to get them to pay for it appearing and they said "no" so you spitefully blur it out? Is that the reason? Or is it some incredibly dumb legal thing? I'd like to know.. in the meantime I'll just keep shouting "I KNOW that's a Mac, idiot!".. why are you looking at me like there's something wrong with ME? I'm not the one who doesn't know what a backslash is, dammit!
Ok, you are a switcher. You used to be a Windows user, but you've seen the light and moved to Linux or Mac OS X or BSD or.. well, whatever it is, you are in *x land, baby and we're glad you are here. But just like moving into a new town, you need to learn your way around, find out where things are and how they work.. you need to get yourself oriented, right?
OK, first things first. There is a book. As I don't know what you might have switched to or from, I can't be specific here, but I guarantee you there is a book. I've reviewed a lot (I mean a LOT) if *xish books at "/Books" here, so you might want to poke through some of those, but if you don't find it, that doesn't mean it doesn't exist - I just haven't read it. It is out there, and you probably should read it.
But having done that, what next? You are competent now, you know how the Window system works, you know where to find the admin tools, but how does this puppy really work? Here are a few things I do:
Hunt for shell scripts
A surprising amount of *xish stuff is written as simple shell scripts. I found one hundred in /usr/bin on my Mac OS X box, and more than that in /usr/bin on my Linux server. Here's the thing about those scripts: they can teach you stuff. Stuff about programming shell scripts, definitely, but also stuff about how the system works in general. Here's a little script that will find those scripts for you:
#!/bin/bash IFS=":"; set $PATH for i in $* do file $i/* | grep "Bourne shell" done
Note: your "file" may not return "Bourne shell script text executable" as both my Linux and Mac systems do. That's OK, just figure out how it does identify scripts and grep for that instead.
I am curiously tabbing
And so should you. If you have bash and are using it as your default shell, try this: type "t" (or any other letter) and hit TAB twice. Ooops, what was that? Well, just a look at every command in your PATH that begins with "t", that's all. Do you know what each of them does? No? Then let's try to find out - for example, on Linux or Mac, you'll find "tack". What is it? Well, "man tack" will tell you that it's a "Terminfo Action Checker" and if at this point you know nothing about "terminfo", that might prompt you to try "man terminfo" or "pinfo terminfo" (and you might read Termcap and Terminfo here also). There's no telling where poking around with TAB might lead you.
Strings and stuff
Sometimes "man" and "pinfo" will let you down. "We regret to inform you that we have no information available for your inquiry" - well, no, they aren't quite that polite, but they aren't going to help you. Maybe oversight, maybe it's just not something you need to know about.. but hey, maybe it's a shell script! Often undocumented little thingamabobbies are shell scripts, and if so, looking at it in an editor might help give you a clue. If not, don't panic. Sometimes it's a link to some other command. Use "ls -li" to find out how many links it has - if it's more than one, go find its clones with "find". Let's say we find something with two links and then "ls -li" tells us that its inum is 56789. If its twin is right here in this directory, you'll find it almost instantly with "find . -inum 56789". If not, you'll need to expand farther, searching the current filesystem for that same number.
But maybe it isn't a link and isn't a shell script. Well, sometimes Google can help figure out what it might be, but so can some tools right on your machine : doing "strings mysteryfile" can sometimes give you a clue (hint: "strings `which mysteryfile`" is quick if you aren't already sitting where this lives). Often just a simple "mysteryfile -?" will spit out a help or usage message. But maybe it's very stubborn?
Well, if this is a Linux system that uses RPM, there is a secret weapon you can use:
rpm -q --whatprovides `which mysteryfile`"
Note that this little line is also very useful when you know something needs updating - it tells you which RPM you need to go find.
There's something similar for apt based systems:
dpkg -S `which mysteryfile`
Source it
Not always possible, of course, but you MIGHT have the source for this little bugger. Try a "find / -name mysteryfile.c" if Google hasn't been helpful. This has lttle to do with learning *x, but I've gotten lucky with mystery applications more than once - the long absent original programmer had been kind enough to leave source that I could find.
Step it
If all else fails, try running it in a debugger. You'll need to know a bit about system calls and all, and when I earlier suggested running "mysterfile -?" you surely wouldn't want to do that as root? Well, you wouldn't want to do this as root either - just in case.
Give up?
No, not quite yet. There's two more possibilities. The first is trying "ldd" to see if the library dependencies might give us a hint. If not, my final clutch is to see if this file is referenced in any other binary or script.. that may involve a bit of work with "strings" and "grep", but once in a great while I have found good clues from this.
And finally: break it
If all other wells have turned up dry, I have one more trick up my sleeve: rename the darn thing and see what breaks. That's pretty drastic, but if you've been through everything else, it's unlikely that this is system critical. Renaming it will obviously break something, somewhere, although it may take you quite a while to find out what. If it really is important, you'll find out pretty quickly - maybe from a phone call if other folks use this system too, maybe from a message in a log file.. but you'll find it.
And then you'll know what it does, right? Isn't learning fun?
I'm looking for some reader input here: we're planning a little cross country trip later this year (we figure on going once gasoline gets REALLY expensive) and I'm going to need Internet access if I hope to make any money while we're breathing diesel fumes.
So: at this moment I'm thinking Verizon ($60 a month for a two year contract.. oh well..) and either the Novatel Wireless USB727 or the Aircard 595U. Any experience y'all have with Nationwide access with these, erizon or anything else would be deeply appreciated.
USB is best for me because we'll be packing both my MacBook and my wife's sorry Acer PC and I'm not buying a card for each of them.. this vacation is going to cost enough already and while I at least have occasional use for wireless access, my wife's computer ordinarily never leaves the house.
So: comments and suggestions appreciate, thanks in advance.
This month's topic is Security Information Management (SIM) solutions, and some of the emerging security information management options that are available with them.
It all started with a couple of security log management applications. Pulling log data from switches, routers, firewalls, and databases may seem a fairly mundane activity, but it's also become a very critical one. Now it seems, SIM vendors are ready to take over your entire enterprise. IBM, Symantec, TriGeo, LogRhythm, EMC Focuses enVision, ArcSight and eiQnetworks to name a few, have all thrown their hats into the ring. Why is SIM, a market that appeared to be all but dead at the end of 2006, suddenly so hot again? And what should you or your organization know about this solution?
After several years of focusing on compliance and breach management, many Fortune 1000 types are now looking to automate compliance and cut costs. At the same time, many businesses are looking for ways to assess the costs and the benefits of security, leading to a new emphasis on risk management. Risk Management has replaced compliance as the action item that organizations are talking about, regarding IT security. And this is the precise area that SIM tools are uniquely qualified to handle and assist you with.
Some of the vendors are even stretching risk and compliance management into the much broader concept of IT governance and the establishment, monitoring, and enforcement of IT and business policies across the entire enterprise. The acronym GRC (governance, risk, and compliance), has become a hot buzzword not only in security, but in business. Some of the larger vendors, including IBM, have created business units dedicated solely to GRC. SIM tools, which evolved from the old system log file analysis applications still used by many security pros, have the ability to track, store, and analyze data about "events" in the enterprise network. Historically, SIM products have been used primarily to detect and determine the source of suspicious behavior in enterprise systems, but many vendors have extended that capability to include detection of any policy violation, including compliance and non-security events.
Other vendors are positioning their SIM products with less features, but with greater depth. Arcsight, for example, is adding the ability to not only track events in the enterprise, but also to identify the business role of the person who initiates them. With this approach in mind you are not just viewing security events, but are tracking new compliance problems, and will also do some benchmarking on how the organization is performing against your existing policies and asset controls. A roles based approach helps the organization monitor not just how its systems are doing, but how its employees are currently using those assets. Still other emerging security management solutions work more at the lower end, helping administrators to set and enforce policies at the perimeter endpoint.
So with so many divergent approaches to SIM and security management, here are some simple tips for a solution that fits your needs. The one that you will choose will depend on which of those functions you will need the most.
1) Look for a tool that can help set policy;
2) Look for a solution that can enforce that policy;
3) Find a way that the SIM can analyze it;
4) And most importantly, find a tool that can monitor it.
Some platforms will likely be used primarily for setting and enforcing anti malware and malicious software policies in workstations, which will be strong in identity management and access management. Traditional SIM tools that do event management and log file analysis, on the other hand, are better at monitoring and measuring policy compliance and risk. Tools that do real time event reporting and correlation can be very useful for monitoring your environment, while tools that do more historical analysis might be more helpful for measurement of compliance, or for predicting future trends that might indicate you're about to go out of compliance. In either case SIM tools work best as a means for benchmarking an organization's performance against security policies, rather than as a means of warning the company of new or potential threats against it.
There you have it. Many organizations are looking towards SIM technology to protect their corporate assets and streamline their IT operations. Beyond technology installations however, deploying a SIM involves an overall operational challenge that cannot be ignored. This will drive what controls are required to manage these risks in compliance with the level of diligence that is required by the organization. The original intent of SIM tools, were that they would help you spot threats in real time. That conceptual ability did not quite work out. But if you look at them as a way to monitor and measure your current policy compliance, they can do even more.
This book is very simpilar to The Missing Manual: Mac OSX Leopard Edition - much of the directly Mac related content is exactly the same in both books. However, it's not just a cover change and a few paragraphs artufully inserted: there is Windows switching information available here that is NOT in the other book.
I think David and O'Reilly would have been smarter to set out to make this two really separate books from the start. Certainly both these books are too big: if you stripped all the Windows stuff out of the first and stripped all the basic "this is how a Mac works" stuff out of this, you'd have two volumes of more reasonable size. However, that's not what they did, so there is a lot of duplication. If you are just a Mac user, you need the other book, not this. If you are a recent switcher, this is the book you want.. though you may want the other also as this does not have everything that has..
This is the book I will give my wife once her XP PC finally dies - I have high hopes for that: it's been shutting off unexpectedly and making grinding noises now and then. The minute it does, I'm buying her a Mac and handing her this book.. that will be a happy day indeed.
She'll enjoy the book, too. David writes well, keeps it interesting and amusing, and knows how to explain semi-technical subjects well.
Apple's 3.1 release of Safari is available for download now (Mac or Windows, don't ask me why they don't put out a Linux version). It's advertised as "The fastest web browser on any platform", and I'm not going to argue the point - it certainly snaps right up ready for work much faster than Firefox on Mac, coming to life after two dock bounces versus six for Firefox.. on Windows the story was the same: about six seconds for either Firefox or IE to be up and running, less than three seconds for Safari.
Once running, Safari continues to be quick: http://aplawrence.com/oneday.html loads in about 70% of the time Firefox needs. However, for ordinary pages without humongous tables or other big stuff to load, any difference is too small to notice. I suppose that as so many web pages are morbidly obese nowadays, you might actually gain a significant amount of time over the course of a day of browsing.
But does fast mean everything? On Mac, Safari displays on odd glitch: the little FeedBurner icons that display in the left sidebar here are invisible with Safari. They do show up with the Windows version though.. very odd. Other than that, I couldn't find anything wrong, and it passed the Acid2 test on both platforms.
There are other Safari oddities: no toolbars, for example. And this odd "snapback" thing which they think is incredibly useful but I just don't see it.. fine, yes, it could be useful now and then but I can't imagine using it very often. Then again, I can't imagine using Safari often either: Firefox has the add-ons (including the toolbars Safari disdains) that I need.. I'm staying with the Fox.
The Windows download also offers to install Bonjour for Windowsw, Apple's Network Discovery thingy. It also brought down iTunes and QuickTime.. I let them install, but as all I use Windows for is to help me lead folks through clicks, I'm not likely to use those.
So why no Linux version? (I said "don't ask me", but that doesn't mean I can't ask you). Apple, Apple, Apple: the Linux folks aren't exactly your BFF's (Best Friend Forever), but helping them hurts you know who.. so whatcha thinking? (I know, now I'm asking them.. pay no attention to me).
We first discovered Hannaford in Western Mass. many years ago. We loved it immediately: they had the foods we wanted and their prices were better than the big name stores. We wished that they had a store near to us.
When we moved down to Middleboro two years ago we were delighted to find a Hannaford's here. It's a smaller store, but we find what we want and again the prices are good. We really like Hannaford.
However, we don't even know if it really was a "data breach". If Hannaford doesn't have a strong CIO, I certainly don't trust that the President or VP of Marketing has any real clue as to what really happened. For all we know, this was an inside job: someone inside their data center could have passed credit card info out or arranged an open door. This could easily have been an "invitation" rather than a breach.
Hannaford's day of shame will pass. They'll hire a CIO or at least a good outside consultant and they will shore up their defenses. But what worries me is that there are a lot of "Hannafords" out there: companies who are large enough to have data worth stealing but small enough that they may not have good security controls in place. I could spit out a few dozen names without even thinking hard: you probably drive by many just like this every day. Small chains, often regional, competing hard against their national counterparts: how many do you think have strong IT departments? I'd guess that not many do.. and that worries me, particularly as we slide toward economic hard times: when the going gets tough, criminals have even more reason to look for prey, and isn't IT often quite vulnerable to layoffs and cutbacks? You betcha: the VP of marketing probably sees IT as mostly fluff anyway.. they don't bring in money, right?
My bet is that we'll see more of this.. unfortunately.
I wanted to gather some stats on the activity of the Premium Consultants listings. It's easy enough to get the raw activity: each passes through my "conlinkp.pl" script (that just records a little bit and does a "Location" to send it on to the real link). So to find out clicks on "conlink.pl", I can just "grep conlinkp.pl access_log".. well, except these all aren't clicks. A lot of them are search engine and other 'bots tracing their way through my pages.
I could easily stop the 'bots from accessing those links, but I really don't want to as there may be value in their finding the sites. However, those are not people, so I'd like to filter those out for reporting.
Well, one way to do that would be to have a list of 'bot ip addresses, but that's a big, big list and is constantly changing. A better way is to look at something 'bots don't usually care about: Javascript. Unfortunately that's not foolproof either. However, 'bots should look at "robots.txt", so if we filter out those also, we should have what we want: real users (maybe).
However, that's not really my point here. In the process of playing with this, I constructed a fairly long command line and realized that breaking it down could be helpful for those of you just leaning your way around Unixish shells. To make it easier to read, I broke it down into one command per line first:
Now let's look at that in detail. I'm going to show a few sample lines from each step of the pipeiine so you can see what actually happens each step of the way.
That's simple enough, right? The "grep" just pulls matching lines from our web access log. Nothing complicated there. I've shown just six lines from the output, though actually there would be hundreds.
By running that output through "uniq", we cut away the duplicate ip's. Again, here we only show four, but the original hundreds of lines would be down to about 150 in the full output. So what do we have now? Just a list of ip addresses, each of which had looked at "conlink.pl" at some point. The next line starts to get interesting.
This line produces a lot of output and is a bit tricky to understand. What it's doing is finding every match for every ip address the previous commands have produced. The output is every entry in the log for every ip that includes an access of "conlink.pl". How does it manage that from one big list of ip's? Well, there are several ways I could have done that, but here I used "xargs". Xargs is normally used to make commands more efficient; for examples see How can I recursively grep through sub-directories?" and Using xargs. Here, we're using it for a different purpose.
The first problem is to limit xargs to invoking grep with only one argument - normally it wants to use as many as possible. The "-n 1" tells it to do that. Next, we need to rearrange the command line a little: if we just used "args -n 1 -J grep logs/access_log" we'd end up with grep beimng called like this:
and so on, and that won't work. The "-J foo" provides the magic we need. We can see it at work if we momentarily change our command to subsitute "echo" for grep; the result would look something like this:
Another way to see what xargs would do is to use "-p" in the command line - xargs will echo each invocation and wait for you to confirm with "y" or "n" before proceeding.
The choice of "foo" is arbitrary; you can use any word at all to act as a place holder. What happens is the "foo" shows "xargs" where you want its input to appear in its output: you are controlling the command line it builds. This gives us what we need.
| grep ".js HTTP"
The next three lines are going to filter this output back to a smaller set of lines again. We're looking for only those lines that have ".js HTTP". Let's review: we found the lines that referenced "conlink.pl", we used the ip addresses from those to find all accesses, and now we're grepping out only the ".js HTTP" lines. Couldn't we have saved a step here?
Well, yes, but the quoting gets difficult. We want something like this:
I could solve that by writing a script that reads stdin and constructs the command line I want, but this isn't about writing scripts, so we'll live with the inefficiency - after all, if I were really concerned about how long this takes, I wouldn't be using command line tools at all. I'd write a Perl script to do the whole task.
The next two lines should be understandable as they do just what we did before:
| sed 's/- .*//' | uniq
We're back to a simple list of ip's again, but now it has been filtered down to only those ip's who accessed "conlink.pl" but also accessed one or more Javascript programs. Finally we go back to the logs once more to extract the original lines:
This is just a repeat of what was done earlier so you should understand it. If not, use "-p" with xargs to follow along. The end result is a listing of the actual "conlink.pl" lines where the ip origination had also accessed a Javascript file.
Now remember: this actually isn't a useful exercise. Some bot's can and do access Javascript and this pipeline would be very slow and clumsy to run. The purpose here is just to show how command lines can be manipulated with xargs, sed and uniq. To actually do this, I'll use a Perl script like this:
#!/usr/bin/perl open(I,"logs/access_log") or die "access_log $!"; while (<I>) { chomp; $ip=$_; $ip=~ s/- .*//; $isconlink{$ip}=$_ if /conlinkp.pl/; $isrobots{$ip}=1 if /robots.txt HTTP/; $isjavascript{$ip}=1 if /.js HTTP/; } foreach (keys %isconlink) { next if $isrobots{$_}; # Not in robots.. next if not $isjavascript{$_}; # and did get javascript.. print "$isconlink{$_}\n"; }