Wednesday, November 09, 2011

Job Hunting on the Phone!

Now it comes to a very squeezed point in time for me as I come to the end of a contract myself and really want to get into a permanent position. I am on the phone, a lot. You will need to use this apparatus, and consider e-mail a poor second fiddle.

Dilema

So there are two dilemas in my and many other's situation : one: short time to get a job;  and two :what positions are likely to be permanent in this on going and even escalating financial crisis. ( An expression now more frightening that what it was chosen to replace :  recession)

Getting a job in a short space of time means being very focused actually on the few very best opportunities you have or are just advertised, and being more flexible - in fact really open minded.
Second to this, and only in that position, you must really consider a "wait out the storm" job while you continue your search.

Good Opportunities now , today in the finance catastrophe? 'You serious?

The best opportunity can be created by a particular counter resessionary industry or niche: for example I applying full out to the oil industry, especially to companies in drilling because there will be more off shore prospecting the the next 18 months than at any time in history. Companies are doubling turn over this year into 2012.

So they are actually one area quite desperate for folk. Other areas could be to do with new legislation in your country, such as environmental ( eg demand for public suppliers to be ISO14000 certified- get into consultancies who do this or go on a course!)

This is good then, that I have some industry experience and that people in permanent jobs are getting too scared to "job shop" - in fact when the recession turns you may find that you compete with MORE candidates of a higher calibre and get ignored if you are reaching a little high for the position. Now is the time to strike into something which stretched you a bit. So a growth industry or niche has fewer qualified candidates but many more "rubbish" applications from people wanting to maintain their welfare/social benefits by making applications. 

You have to stand out, so firstly you have to get your name known.

Postioning Yourself with the Initial Enquiry Telephone Call

When at a firm there is a contact number and name who will be the direct manager or departmental one over the  porpsective employee, then you have to phone them. Firstly use a work line or hide your mobile number so that if it is an answer phone you can drop the line : do not leave answer phone messages- you want to get them direct, on the spot.

Having said this, above all be super polite: say you are really interested in the position and do they have a couple of minutes now to discuss it with you? If not suggest a better time, and then you can choose by the tone of their voice if you leave your name or not, or ask them to call you back. It is best to cut it short and try later.

You want to prepare for this- write down questions in large, clear text to read from : one with some opening yes/no questions: "will this be a probably permanent position?" for example. Then pick some of the wished qwualificvations and ask which are actually most important for this position because this is often actually not clear: it could be IT seals the deal, or perhaps a pleasant outlook on life!! If they are going to call you back, then keep those questions rolled up in your pocket at all times !

 Ask then most importantly, when they want someone to start- this is your first positioning strategy , your USP over most any employed person and marks you out as not just a  "job shopper" . When they ask you what you are doing, say that you have been in this type of job or have relevant skills and that in principle  you can start tommorrow because you are available.

Do not necessarily ask about pay. Employers in non union / tariff countries can pick and choose now and abuse people a bit. Pay level will go to the wire, and you should take it then, holding your ground for an offer which will actually loose you money over time. Ideally you should have an offer for a competing job, but that is maybe unrealistic in todays economic climate!! Alternatively a  local, low "on-cost" job to go to by the time you get to offer stage, gained by the  boost you have just had from the interview round ! 

Be prepared to say "No" to a low offer which will for example, put so much transport or relocation expenses your way that you will go into the red!  But DO NOT SAY NO right away when you are offered it. Always say you need time to think about it. For a job which involved relocation, this should be over a weekend or a week so you can discuss it with your "significant others" and look at the lease market in the new area. If it is a low offer, hold them off as long as you can and then say they will have to help you out with transport or relocation.

Also as blogged before: if it is an agency then go direct and say the wage you are offered is lower than discussed face to face: it can be the agency are skimming too much of your out-charge !! Also the client wants a solution from the agency and has a big lever over them to take a lower fee on top if they use them alone. This is why it is important to swap business cards with the actual employer or note down name, department etc so you can reach them through the central board.

In the time you hold them off, find out what the going rate is for this job. Also ask again what training is actually going to happen and how soon into the job? If they say they train, but are vague about the course date then be suspicuous for a low paid job, especially in sales and market research where training involves a period of someone looking over your shoulder on the job, rather than real lectures and exercises.

If however, it is a going rate or very good offer then DO NOT LOOK A GIFT HORSE IN THE MOUTH unless it is in a place with the highest lease and property prices out. Say in the phone that you accept in principle, just pending the documents coming.

For a job  you are unsure about, ask for a full contract to be sent. This can be vital to show if in fact when they speak about overtime, if it is paid or not, and if there is some underhand ways of paying you less or taking money for uniforms, materials etc. In many countries this is your right, or within 2 weeks- so they may as well draw one up. It is a good stalling tactic with a job less than ideal for you.

What Can you Negotiate?

Having stalled them off over a weekend and up to a week, you can then start to feel out if you can play a little hard ball or just get some compromise out of them. FOr a job where you will loose seriously out financially for no other reason than taking that job, you can say outright it is too low and then suggest a figure nearer your goal, and then hold in reserve that this is based on a 37.5 hour week only, over time expected.

What is realistic to negotiate with jobs you have to relocate for is that they cover that relocation fee, at least for yourself if you move first without the family. If this is not on the cards, then you have to ask if they promise on a personal basis, to get  you lower-than-market average price on accomodation through someone at work. This is a diffuclt situation, as you really want to avoid going into debt for a new job in the shakey economy we have, whereas you want to keep on working. One stalling tactic then is to accept the job pending suitable accomodation as a foot note to your letter of acceptance. An additional strategy now you should employee is to postpone the start date up to 6 weeks so that you can find affordable accomodation.  In the USA most people have very little if any job security, so if  you do not find somewhere you can afford, you loose nothing in accpepting a job and then just turning round and saying - actually I can't afford to live here, what shall we do?

Is stalling not dangerous?

You need time to absorb and gather new information: This is actually the real reason to stall off from an immediate "YES YES YES!" - you have time to reflect and find out what people around you think


You may find there is another solution: for example you may have a relative with a room in the area: someone may already drive to that company or industrial estate from your home town; you may want to work a four day week to keep one day to look for a better job. You may find you can take an express service to beyond the location and commute back counter the rush hour traffic by bus or in a little cheap car you leave at the station each night. You may find there is a motel who do long term rates cheaper than the lease market, or that you can take it on that "out of a suitcase" basis for a few months just in order to get the experience and keep on lookign for work.

Remember all  your costs are taken from post tax pay, so you need to earn 30% more to pay for them! I

If they say "well I need an answer today" on a crumby offer, then use the tactic above- say yes verbally, send me a contract and I can start now in 2 - 3 weeks time.
 
Hard Ball

The tactic of accepting an offer verbally and going back on it is hard ball, and one way play: are they going to sue you? They may well black list you with the agency you went through though, so make sure you talk it out with the agency first. Maybe they can help raise the offer or appreciate you need more income.

Anyway, with an offer in hand you can now put a lot of pressure on other applications in the pipeline just behind this: then there is some SPIN SELLING in that suddenly there is a risk they lose you, so over other equllay qualified candidates 'it is now or never" for you.

Hard ball is also to fib: say you have had a better offer!


Wait out the Storm

In terms of a waiting job, it has to be local to where you live or somewhere cheap to live if you are YFS because it is inevitable that you will not be expecting as much pay as your target career path!

You shouldn't prostitute  yourself with more debt and commitment to expensive leases on second appartments and weekly commuting for a waiting job: you may have to give notice time and be locked out from a target job !

Take a "Trainee" Step Stone Job

Alternatively you can look at the job market and companies you find attractive and then see what extra little competances you could gain in a waiting job. I have done just this- using the new MS version of an  ERP system and moving from biochem into mechanical purchasing in a lower paid job to get forward to permanent opportunities with higher pay than I would expect here otherwise ( oil industry is the only show in town worth speaking about actually!!)

So it may well be the latest software or the biggest/ most common ERP you lack experience on which will make you stand out when you go for a target job and company.

Friday, October 28, 2011

Internshop: The work Placement Dilema

Working for "nothing"  in order to get valuable experience or as part of a course or social charter, is a double edged sword at best. How do you get the best out of one and when do  you need to say " no thanks" to an offer ?

My contension is that you need only focus on two the internship : hard skills and the job reference should be your focus above how glamerous the industry and the company's  brand name. A lot of very unglamerous people work brand names. Brands are changin fot be with todayommorros!

Is internship worth doing? Were Bill Gates or Steve Jobs ever interns? Are you in fact not better off these days riding the back o

I think I have gone thorugh work placements three times in life now. If you count poorly paid practice work, then there at least two more. Do i feel exploited with this being 10% of my adult life? No . i do however feel that I DID NOT EXPLOIT those placements , and I could have had a lot more out of them if I had been more savvy with neotiating and using my time properly, for ME.

So the rists quesiton is: Is this a placement which will lead to a job here?

Well often no. In fact it can actively work against your best effort given as they may be too emebaressed to carry on exploiting you. On one of my paid situations, which was a pittance, so let us call it an internship, I actually did a better job than my management consultant, and was a threat to him so he managed to brush my work under the carpet and put his name on it after I was going out the  door.



If you have a chance of a job, then you are on display and everyone knows that and some will take full advantage of you as the "gofer": cappacinoo round, post office etc. Just like being the noob: but you are not being paid! So in fact you  should be able to negotiate yourself be able to negotiate yourself some barriers to this and areas of real responsibility you can fend off the ( usually  late 20s., early 30s female or male chauvinist middle aged dicks who like dumping from their small height above the shit steam which they have taken 5 , 10 or even 20 years to rise to)
Of course film companies and some ad agencies rely on gofers to do all the menial stuff, and you are expeted, like being  a "fag" at english private schools, to go through it with stiff upper lio and duly pass-it-on to the next generation.
However, I advice avoiding those;: Menial work experience is menial work experience what ever the brand. Are you better to be an intern at McCanns or a branch manager at MacDonalds for your finance-crisis experience job? If you are going to work for free, then you have to get gold worht out of it.
WTFINIT FM: What the F' is in it FOR ME? But also, what can I wangle out of it in terms of knowlegde and bartering or siexing opportunities further.
You have to set out what they will be able to involve you in, what "systems" ( not just IT; but the logistical, financial and legal process that go on in a branch and the cander and not least jargon the people have )
It would be a good principle for real jobs: stipulate what you will NOT do! In fact, the spoilt brats here in Norway do that or try that on, and thus make me and many immigrant females much more attractive for the price tag because we actually work for a living and will fetch an occiasional coffee without feeling demeaned!


 THink not only that you will get one step nearer your dream job, but also that you will get a sellable skill out of it. For example, ERP systems: can you send out invoices and check them in on system against purchase orders-? Can you check and file technical and ISO documents? These are actually well paid jobs in oil, health and research industries! They can be very good fill in jobs post internship and also USPs for you selling yourself into a dream job





In fact your internship may take different perspective on this: do a hard skill in IT, or in say print finishing or writing as your internship in an unglamerous company ( go-ugly-early, see an earlier blog of mine)

having worked in sales a couple of years prior do doing my masters in marketing, I found that more of a hindrance than a benefit, because on the one side it was not relevant to the services and consumer product marketing I wanted to get into, and on the other it made me better qaulified to do a marketing job than many of the marketing managers interviewing me! I had no real focus, and in life, you need focus. You need to have a real lust to go somewhere and then the path becomes clearer, or rather obvious blind alleys become blatantly obvious!
As a management consultant I have advised technology companies not to offer free trials of their products. The reason is simple: if there is no monetary value on a project, there is no loss in shelving it. People's time on both sides are just washed away. So putting a price on a product or technology trial, or service test means it has a value and a loss to be incurred if it is halted.
This is the same for internships: if you are not getting paid you are both expendable and not maybe worth involving in higher risk or longer term projects an customers.
The absolute core of What is In It for Me is meeting people.  However, as a lowly intern you will see more snobbery and protectionism than win-win move on to a better job sitation. Hard skills are always better when you are a graduate, because you have just stooped so low as to be an intern in the eyes of many. In one sense, doing an internship in something glamerous or somethign dreary, which leads to nothing is a loser position and you have brought it upon yourself! You have nothing to lose in negotitating some responisbility and training. Walk away

 In the USA many "glamerous" intenrships are shop windows for the offspring of the mature Ivy League, and pilgim vieux riche. They are there to drop the name of their parents or rich freinds to the industry front door.  So why compete? If you bring no family , friends and Ivy League contacts for the companies to use you a bit more when they give you a job, be wary.
So that is one rule: do not go where there are a lot of interns and some of those are on an obvious rider
What shape? That is the crux now you see : what shape the internship: like my last one, is it to be an 80% position so you can work part time or take the kids to-from day prison? Are you to get some hard skill training and experience in return for making coffee and doing the post run?  What will you sit in on?

The one key thing you will get out on the people side is a reference. THis is the best thing. But remember not everyone in a company needs to be your referee, so why treat them like that if there is no job there. I didn't laterly. You have to impress two people : your boss as a reference and someone higher up, maybe even the MD. You then have to appear industrious to people who influence them, and also not to appear a " facebook" addict to the trash who work reception etc. This is all easy: Suck up, shit down, keep out of trouble, use your mobile for facebooking.

Thursday, October 27, 2011

Job Hunt Again

Having just had a good interview, I know the excitement and confidence it can create and then the fall which can come afterwards.

The best antidote is also the best practice anyway: don't rest on your laurels.

You can never be certain you will get an offer. It is not just other external candidates, but internal shifts and budgeting which can fill the position or actually reduce it from the head count.

One thing that is certain, with two offers you have an even stronger position to bargain your "end game".

So the steps to take are: get feed back on how it went : asking general open questions before getting to the crux of the matter: how do I stand against other candidates? A knowledge of the time span they are doing first or second interview-round you are in is vital: it is best to not let them off the hook with "we have many candidates yet to see". An initial call to the personnel person present, or the recruiting consulant is worth while because they will see you are keen. Also if you are long off the mark they may well tell you then and there.So then you can get on with job hunting with a little bit of humility if you were trying to stretch yourself, or if you were over competant then you can adjust  your sights upwards or undersell yourself a little next time.

When it comes to end phase in interviews, then is the time to ring your prospective manager back. Then you have them in a corner: they have seen everyone, they have come out with an impression on both paper of competance and in person of sociability. Now you have the chance to influence the decision making process.

You should be able to find out your short comings in an objective way. what challenges do you think I would have in the position, or any actual difficulties? DO you think I would fit in socially ? How much better qualified and experienced are other candidates. Be prepared to intersperse these questions with general ones, like how the interview with you had gone, what information may they like to support this , is the start date and so on.

At this stage then you may get a little "puff of smoke" that shows where any shortcomings or mismatches come from. Think quickly: pause and then ask a qualifying, open question followed then by one short closed question. Breathe, then offer a counter point and get agreement to send evidence to back this up.

From this conversation you will get useful feedback from many managers and a feel for your standing at least with managers who go into major avoidance: if they do this then they are covering up for choosing the girl with the biggest bazoongas ! They may just be indecisive or like this situation badly: believe me though, that type of manager is in the minority as in today's flat structured management trees, they didn't get there without being objective and discussing personnel related issues with people.

Move on: do any follow up, and then adjust  your CV if you see the need. Be prepared to adjust your whole approach: be more up beat and up sell yourself, or the converse (as here) be coy and undersell yourself. Also if they really doubt you can do the job, then you should talk around with other people to see if this is just one opinion or in fact, you need to aim slightly lower on an experience making job.

Finally you want them to agree to giving you brief feed back in the telephone if you don't get the job. For some managers this in fact can be enough motivation to give you the job when they cannot otherwise see a clear winner.

Wednesday, October 26, 2011

Managing Recuitment Consultants as a Job Seeker

Intro

Generally of course, I recommend not going through rectruitment consultants, but instead targeting firms you really want to work for, in geographical areas which suit you ,  and applying speculatively or networking your way to openings.

When a consultancy advertises a position I like the look of,  I have often rung them up,  taken the name of the employer, said I am not interested and moved on to a direct application in the guise of an open applicatioon or personal recommendation to apply.  I heartedly recommend this because it can save the employer a lot of wad and it shows initiative and a winner instinct.

This has won me jobs several times. On two occaisions the recruitment consultant did not even want to put my CV forward!! Recruiters being involved in the converse, ambulance chasing positions I am already in line to get, have messed up a couple of serious career possibilities.

I speak very badly about them because 99% of my experiences have been either disappointing, lead nowhere or been outright unpleasant. However, a couple of times in life they have set my career on the right path and one day when I was sitting about feeling a bit sorry for myself I got an odd call from a well known agency, out of the blue but very casual ..." I don't suppose you have thought about working in advertising agencies who specialise in....?". So they can be worth all the hassle just to dig to that one gold nugget which could change your life for the better for ever " 

Some Times, Recrutiment Consultants are an Unavoidable Hurdle.

They are - to  paraphrase- an unnecessary evil for the job seeker, but in many cases they are the only route in through the doors of attractive employers and also they can have a larger hidden market and "quickies" which just skip the advertising cost because you were in on that day and they can send a few candidates immediately.

That is then a good place to start in "How to Manage Recruitment Consultants".  How they and the process operate.

Why Firms Use this Necessary Evil?

Consultatnts are used in large part these days to 1) save clients time in the siv stage 2) manage down wages to new starters. They may have a framework agreement which includes a total saving, a freeze on rises over previous year or percentage under inflation for example.

This can affect even jobs which are agreed to be a higher level at the client interview you have: this happened to a top qualified marine architecht I know, with whom the client company discussed an excellent package and relocation to London, while the rectuirment consutlant made an actual offer at "Price Parity" to his current  wages, stating his motivation to move company as being enough compensation!! Lose lose all round on that one.

Being Human

They are only human and they have often two goals which are actually in conflict:
  • get the employer the best candidate quickly, 
  •  do this for the lowest wage.
The higher the wage, the higher the income per placement because they work a %. However, to get a retainer / framework exclusivity they offer to hold wage expectations downwards. Thus, they then need to spend less time on each placement and in times of high unemployment like now, they can often afford to do this for jobs there is ample people supply for.

To be fair I have had consultants fight my corner maybe a couple of times, spelling out to companies they have to put a company car on the table or what the market rate actually is....but usually they are out to keep to a framework target of reducing or putting a lid on inflation. When the basic salary train drivers earn emerged a decade ago, a lot of "Yuppies" felt very dispossesed as they struggled on less than that with 50 to 60 hour weeks without any over time! Serves them right actually.

So, they have a bit of a conflict in time: they need to send a range of candidates who may work for the lower salary. Further more they need the employer to make a decision quickly so they win the commission and don't spend more time on it. Or worse, lose it to open tender as is often a get-out-clause for framework contracts.  So sometimes they have a prefered candidate they have actually spent time finding and qualifying,  while they think the rest are window dressing to make that one stand out.

 This is where lady luck and your own persistance and skill come in.

When handling a  recruitment consultant it is important that you have some short selling points which are appropriate for them.

Client Centered
Rectruitment consultants claim to either operate geographically, as stand alone or chain-branch-offices, or by industry or job type. However, they are only human and will take what they get from a sales call or their networking: so often they "bleed over" into other areas of just end up being a jack of all trades.

What is important for you here, is that they go by CLIENT first and foremost, often to the exclusion of all other competitors in an industry. They go by who they are pressing the flesh to now and especially now in the last three years of recession, are very opportunistic just to survive.

So this is the most likely situation you will encounter if otherwise, like me, you hate going through them: that an employer stone walls you because they have outsourced all their new rectruiting and bemanning. You  like the employer, you must like the recruiter.

Armed with this and a will to show soveriegn motivation and drive to get on, you can say you love the employer and will die to work for them, but you are better off being a little coy and probing both the employer ( via contacts in Linked in or your other networks) to find out where they may need people, before then having a killer  targetted CV; or the opposite, a cover-many positions CV:

Be Direct, and Direct Your Chi on the Most Positive Point in Time

So to summarise this first approach, the stalking, come only to a consultant with the most poignant of application relating either to an advertised position or a demand you have sniffed out  and that could be you find your skills and experience level are actually in short supply in that area in addition to your sniffing around employers.  This is the way: rectruitment consultants are not blank walls, they are gate keepers at only one gate into the company's castle.

You must be very concise and very pertitenent as a candidate: ie very targeted communication which lights up the recruiter on the other end of the phone, or better still, in person.

I have done some very good general sales jobs on rectruitment consultants which have lead to sweet zero. This is because "nice guy, move on to the next really handsome one" ie the best candidate for the jobs in hand, you are an also ran.

Also Ran

Being an also ran extends unfortunetly very much into the interview rounds when you do get them to bite on your pertinent parts!

Recruiters want to send 10 to 20 filtered CVs (from maybe hundreds of applications- be grateful when they send yours forward, don't look a gift horse in the mouth at this stage)

After this they want to send between 5 and 10, averaging at 6 to an interview: or collect the 20 best at a one of the god awful "recruitment days" ( all 20 got sent home from a really expensive one of these I went to in 1993!, with a very disappointed and presumably fired consultant!)

As an also ran at interview, you do have a lot of chances though: one of your competitors will be just after a pay rise, one will be in a position to trade job offers against each other, one will mess up the interview somehow or turn up late or not at all, and then that leaves you and one other probably roughly equally qualified candidate.  The fact is, Mr or Ms Successful who is best for this position, is probably best for several others so that is actually on your side in pure probabilities, in that they take another job or don't move. Then Mr or Ms Loser has got so far on paper but is a dunce in person.

Can Start Yesterday!

One thing above all on your side as a job seeker, ie unemployed or in a "waiting room job", is that you can start immediately and WILL accept a so-so offer. We can come back to offer  bartering later, but now you should not let yourself get pinned to any money: you should push for job satisfaction and understanding this job and this company is really for you to shine in or not. You are available to keep that fifedoms head count up before the cut comes!!

Pay Strategy Pays

It pays to have a strategy to not commit to any pay level in interview rounds. There agter to not be forced into making a decision between just you and the recruitment consultant with respect to this.

This is the golden rule: establish direct feedback with the employer on salary offered , as they may not know the offer is at the low end or maybe needs reviewing upwards!!!



Play the long ball on the Jobs Content before Wage

A friend of mine recently worked in the public sector as a hired in consultant, in practice actually doing a marketing management job. It came time for him to either come into the body of the church as an employee or they could not budget for his (modest) fees.  He was very coy in not letting them define the job title as in the public sector this relates immediately to pay level and all the equal pay for sexes and races laws which are mostly enforced by jealous coworkers. Instead he said they should define the work tasks  and goals of the job forward to see if he was the right person for the job or if it was in fact a more junior or senior role. Some months later he had then painted his employer into a corner of definging responsibility level which he could then compare to other positions at the location and in the public sector, and hey presto, get the position upgraded. 

This is a consumate tactic or even a strategic approach you could say because it takes time to build it up. It can also help you define your own career goals and align your stregnths, limitations, challenges and need for training for the position.

If anything it is a good stalling tactic, which gives you time to talk MORE about the actual daily tasks in the interview, which is good. I had a two hour interview yesterday, with an advisor at hand, not through a recruiter as per my own preference: there was only 10 minutes on what I would be doing day to day and if that was both  within my set of skills and actually interesting to me.

If they try and pin you down, throw back to them what range they think it will be in. Then say it depends more on the other benefits and training the company offer more than a market level per se.  Finally, you can say, for the information you have it is too early and you can research what is reasonable a bit more in between times. This shows you can negotiate and not be pressed into a corner :

Be prepared to walk away from silly low offers, which happen all the time right now I hear, and also taking a job in London which pays well but is long hours and offer no support for your relocation or social well being.

End Game

There must be an end game if the offer is not "take the large breif case full of used notes and leave the room calmly" which I have had on a couple of occaisions. However the end game is as much with your own enthusiasm and maybe desperation for a job as it is with the consultant.

The fact is that you NEED to be able to live and commute to your new job in that area. If they help with that NEED by putting  you up in a hotel, or finding a room within their network to a resonable price, then grab the chance to show you are good. Living costs will be your biggest outlay and avoiding full relocation costs is pruident. Otherwise, with so much job insecurity as there is now, DO NOT go into many thousand of pounds debt to relocate yourself.

 Companies can write relocation off against tax: they do so for their top executives, putting them up in luxury serviced appartments: they just want to avoid the up front costs and the admin or cultural demand this would place on them as a "social provider" for the hoi polloi like you and me.

My point being in winning over yourself that this job offer needs to be looked in the mouth to over come your enthusiasm and emotional response

Will it really get you up the career ladder ? WIll it just frustrate you? How many hours a week do they slave drive you? What are they prepared to do to help you? Do they care about more than paying your salary at the month end and asking how your hunt for flats is going?

Many people take unpaid internships in the USA and in law firms and ad' or news agencies in the UK just to get into the game. So for a real opportunity, given you can keep costs down by say renting a room from a well meaning coworker at a reasonable price, then be prepared to take a lower wage than you might expect.

I have actually just finished 8 months of placement in a new career direction, and yes it has been financially painful for me and the family, with many sacrifices, but it has been a great insight to my new direction and gives me fresh shelf wares on the CV which are getting me better interviews than I had imagined!!!!

It can be wise for both parties to enter a short trial period where they pay your full hotel costs and arrange commuting travel with a hire car or coworker. Anywhere from two weeks to two months or more to see if it is a good "fit" for both sides.

 I have had this once, and it was totally win win, and I got my usual wage on top of a B&B with a nice rural english pub up the hill!!! ( Chruch Inn, Uppermill to be precise!!) In fact I wish I had done this on at least two other occaisions because I could have said no to the boring work or aggressive management style.

Do Not Miss A Trick by Not Asking the Employer

  Many companies are only too willing to help you out big time, just take it up DIRECT in an interview with them because it may be percieved as a trip wire by the consultant, or a way of locking you into the need for the salary by the high cost of relocation, which is pretty nasty and I have come upon this. Usually it is benign neglect on both the employers and the consultants part, so put it there on the agenda direct with the employer, even if you will be in the intiial stage legally employed by the consultant acting as a bemmaning bureaux..

At the end of the day there is the final offer and the final end game. You should ALWAYS say you would like to see a written offer in the post and then you should then say you need time to think about it and talk it through with your significant other(s). If it is a good offer with good side benefits, don't hesitate and just talk about start date and relocation ( which is likely why you get the job- willingness to relocate is a biggie !)

For a mediocre or outright rubbish offer,  you have actually a lever and don't get all depressed. The difference between 15 and 25 grand a year is about 900 pounds a month with social costs included. But the difference between a rented flat which is el cheapo found by the employer, and a normal market price could well be higher so you save that difference !  Don't forget that your costs are post income tax, so they are all in fact 1/3 higher to your gross income than they appear on paper (unless like here, you can write relocation off against personal tax)

Your Leverage in the End Game ?

The lever is that you are now the only car on the track. Now you can put the foot down a bit with an open goal, rather than just roll in over the finish line.  Now you can take a few days thinking about it, and then let them call you, not the other way around.

Stall them off again, up to a week is quite okay from original verbal offer. Stall them on relocation, talking direct to the personnel department on what can be offered or what is normal or special needs for you. Say you are waiting to hear back from the personal department.


Buying time puts more pressure on them, not you, to change the stakes. Other candidates lose interest ( the offer round is why you hear nothing back for several weeks when you have had even second interview) or get other jobs, or will not like the package which is now on the table and budgeted for. Time drags on a bit, the employer wants to avoid going through the whole 3 to 4 week process again, and the consultant wants their fee or the start date for their reward to come after 6 months. Each thousand sponbdoolics a year, is only a hundred or so per calendar month for them to cough up including social taxations.

Having pushed it near to the wire of how long you agreed to reply to the consultant,  cut out the recruiter: go direct to the boss who will employ you and discuss any possibility for relocation expenses or a higher wage: if they say no, then say out right, sorry it is too low a wage I have other options I would like to take up then instead: is that your final offer or can you better it?

Sometimes the manager may have a biudget to use, an idea of the market rate, but  not actually know what the recruiter has  actually offered. Rather the reverse the recrtuiter presents a cheap fait acomplis on a plate to the manager without ever raising with them  that your nose is well out of joint on the low offer. Presented with "we offered x and they have said yes!!" no manager in the private sector at least, is going to then say, "oh let's be nice and pay them more anyway". 

Be prepared to just say no for a low offer on a job which will just cost you money and time. "Is that your final offer? No thanks, it is too low. Goodbye...." hangs up telephone. It can be worth doing this on e-mail to keep a cool head and see if they ring you with a counter upped offer.

At the end of the day you may be able to say to other job interviews that you are under offer and would like a decision for second interview later on that day or not to count that new opportunity in or out. There is the pressure thing again: use them or lose them. Simple human reaction: something is suddenly salient and worth more than its actual value by nature of this limited availability.

Conclusion : The Reach Around

The Key to managing rectruitment consultants as a job seeker is to ignore completely their desire to be the interface between you and the client. See right round them,  from when  you get the clients business cards and present your own business card at every opportunity for "come back" out of contract. 

Even before that, make direct contact and then act suprised when they say they actually use Recruiter X and okay, now I will go through them but perhaps you could point my name out to them ( in a follow up  mail for example)

Establish direct contact outside the interview, and take control of the flow of information. You need to know all you can about the job, and match you abilities against it in communication and extra evidence which will be diluted by the consultant. Also raise any doubts early in your own mind on your motivation and perhaps short falls relative to the job.

Lastly,. when it comes to an offer, go direct for the conditions and support benefits around this and preempt it with some time buying and more excuses and decline the offer directly if the worst comes to the worst.

Wednesday, April 20, 2011

Wes Anderson, Pixies and

Wes Anderson has kind of "snuck up " on the conservative trogladite critics. A couple of cookie films which in the US floated with Bill Murray. That should have been the clue for the sleeping moral minority: Anderson Co. had won over stars and funders to run with them.

Conservatives want something they understand: they don't like change, they are neophobic by the very name they adopt. Indie, in music or film, is a convenient label for them to try and talk down to the market about.

Well the market is a funny thing when it acts freely: when enterprise in holywood or motown takes a chance on somethign creative and hey well, the market is unpredictably positive. In films, Wes Anderson probably tested well in pre-screens which gave the green light to mainstreaming his wonderful films.

Indie is no longer indie when it signs up to the big labels. Conservative, neophobic forces are thank fully the laggards in the market and not the early majority.

Friday, April 08, 2011

Data is data- Using an Excel Trick

Okay well I did say I would blog more on the nuts-and-bolts of data, but for now we can look at data its simple form we can relate to: in tables- rows and columns. In earlier days tables were called "arrays" in computers which simply meant that there was a column and row "cell" address structure and values / text could reside in those cells and therefore be referred to or manipulated systematically. (These simple arrays are still used today, for instance, in XML often "on the fly" data-shutteling in JSON.) Of course for the last twenty years we have all been accustomed to the king of the table: Microsoft Excel. Now because data in larger databases, or on for example tables or lists on web sites, can be exported to excel ( often via ".csv' files- comma separated values) or directly pasted in we have the opportunity to "slice and dice" or grab information and use it quickly. You can think of database data as being lots of simple excel tables, but all being connected by common denominators like unique user ID, street address or so on. So here we have the first two tips- check there are no commas in the data itself ( choose tab separated if so! ) and use paste special > values to paste in from any HTML or word etc formatted source. You find this in right click and now there is a short cut to plain text values. Why would we want to export to excel or paste in tables? For example, we may want to compare listings between an old and new database to ensure completeness and non duplication. Alternatively we may want to be able to summate, search, sort and work up a table we find on a web site. This is very common in say "migrating" customer listings from an older system to a new database. On a more complex case: we have two databases running on different computers: we have a list of flights from a travel company exportable to csv, we have a list of cost codes and budgets in our own ERP and we then find a web page with listings over room prices at near to airport locations. We want to calculate a cost. A case where we recieve two excels would be in a tender process, and these may be far from immediately comparable: in effect they are two different "data bases". Just as in the cases above, we need to combine the data to make for comparability and cross refernce. Steps okay so we have cut and pasted in clean, unformatted data or opened a simple table file in excel. Cowabunga: our old database we have in excel output has "Mr John Dow" in one field whereas the new has Mr - John - Dow in title/firstname/surname fields! Okay so we need to do a text to columns, but we may want to do the reverse : merge the three into one: ideally we should have the three but retain the single field for common sense checks when an operator in the company is faced with a name they cannot interpret. Text to columns is in DATA. Choose either fixed, which gives a line between characters when all are the same width ( useless for peoples names!!) or tab or comma delimination: you get a preview to scroll down to see if this will cut well, or disect up your names into garbage. If this fails you can say replace all Spaces "" pipe ( CTRL H is find-replace shortcut) and then export to unlimited columns: then you can clean the data later easily by running a sort on the outer columns to merge them back in. Here then comes in another fundamental: using SORT you can find empty fields, wild outlier values and plain old garbage so as to clean data or get an idea on how dirty it actually is!! Of course this can show you that your preparation work has had a bum-steer and you need to rethink your approach. Now another very common thing to do is to have two different tables with one nice clean column of cells in common. This is preferably unique: ie values are not repeated/duplicated. Now we can apply a very simple database command without having to go over to setting up the straight jacket of Excel databases and learning all that jazz. We have the VLOOKUP function: this does what in SQL is: GOTO-tableX-ColumnZ-Find Value Y: report back Value in Column Z+1 value. So you can lookup data from another table preferably in the same workbook on a different worksheet so it is not lost ( you can link to other workbooks) or pasted over in the working area. What VLOOKUP depends on is that the data is the two most leftward columns of your selection and that the first column is preferably ascending numerical values. You can therefore cut and paste the column you know you will find the unique value in over to the LHS and then insert another in column A and draw down 1,2,3 ....until you reach the bottom of the record. The first part of VLOOKUP then is, using the "wizard" thing not hard programming per se: Cell Value to Lookup in the other table: this is best to have as a fixed column with '$' if you want to draw out several other column values from the search: eg date, monetary value, total, etc based on looking up "name" ( use F4 for inserting dollar sign $ to cell references_ it toggles both-columnonly-row # only btw) The next is the table array to go and look for this value in: here YOU MUST make it absolute and this should really be default because as you draw-paste down the column, you $ both columns and row numbers. Otherwise the table area you look in will move with each cell you draw the formula down into. Next there is which value to return by colum : this is tedious, it is the number of columns in the selection so in this case "A" is one, B two and so on: it may be you use column F to make for a clean data search avoiding some noise to the left, so this then is your column one. Usually you would want to return the value of interest, but you can also use VLOOKUP to look for exahustiveness and fidelity between lists on the same supposedly identical value. Finally the logical operator as they call it is always "FALSE" for some reason which then send back the data. True does not for some reason?? Okay, so now you should see returned data, and maybe some "#N/A" valuse: n ot avialble. If everything is NA then you have to check the formular: or if NA starts at one point, you probably have forgotten $ ( F4) Errors are usually caused by: not having a left hand column in numerical having hidden spaces or tab characters in the lookup column having confusing data there being duplicates ( the first one is the only one reported back on)

Thursday, March 24, 2011

DIY Social Media Sentiment "Barometer" Techniques

Contention: . Traditional hard and boring market research techniques can save your bacon when you are looking to turn-around fast, accurate and most of all PROFITABLE reasearch reports or data mining portals.

I've been lucky to be in the doors of a couple of Social Media Monitoring hot-shops (SMM) and also have a friend who has programmed both back- and front-end for screen-scraping and data mining in web monitoring resources.

I see that today's technology means the usual IT smoke-and-mirrors dissolve away into what is today pretty standard undergraduate project work, just lots of it. Even sentiment rating tools are at a level of final-year project for a Harvard Comp Sci Major.

In this blogg let us look back under the bonnet (the last blogg) and maybe chuck out the v8 single carb for a nice little supercharged injection engine.

A Heavy Handed, Verbose Approach to Social Media Monitoring

What the media-monitor companies seem obssessed with is having accurate census of all posts in consumer forums, and now with access to FB and twitter, all hits on the brand names. This is because, like Everest or the Great Wall, they can build it. Not that they really need 'go there' but because they are used to huge data resources, it seems the right thing to do.

Now this is by no means a labor-of-sisyphus, it probably just feels like that, espeically when major forum web sites kick your crawler's IP address off their triage-server. Up until now, server resources have not been an issue, but with capture and indexing for Twitter, Digg and not the least FB indexing and retrieval navigation start to become sluggish and resource heavy on both vertical, horizontal and labor in maintainance and expansions.

So faced with the practical, economic and client-patience issues of providing huge data mines, what is the alternative "blue sky" out there? Well of course come on guys, don't re-invent the wheel.

Comp'Sci'Bachelor's Myopia

There is a cultural issue with computer science "majors" because they race through math, especially statistics and only listen to the more web oriented parts of any ( if at all) marketing course units. Like many people outside the profession, it seems all comms and "fluffy", not a serious branch, just a bunch of luvvies making hot air.

What they all want to do now is either be a hot java programmer, admirably, and/or do phone apps or APIs for web sites. What they do not want to spend time on is "aligning" to the harder side marketing. The web is the new patch, keep off with your Taylor Nelson 1950s stats and punch cards.

Well post-pubescent-programmer, er sorry, software engineer/app developer, you are wrong. Traditional hard and boring market research techniques can save your bacon when you are looking to turn-around fast, accurate and most of all PROFITABLE reasearch reports or web.

Sample Dear Boy!

The first population cenus was perhaps conducted and documented as long ago as 3400BC according to Wiki. Census means getting ALL the possible data points in the bag. Even in the infancy of market research, the census approach was rejected for consumer reports because it is just too combersome and expensive, and if you make any mistakes in the methodology, like ambiguiities in questions, then your entire results are biased and not accurate. Also there is just a lot of irrelevant data capture points.

Market researcher managers, the hard-men of the industry, soon adpoted sample-set techniques from statistical science used earlier in biology for example, and went on to develope their own methodoologies for creating such smaller sub groups. The key here is that the mathematics gives you a known accuracey or probability that your results from the sample group represent the population - as whole say of a country- or as a set of say, people visiting Wallmart on tuesdays.

Researches uses Random Probability Sampling and went on to use stratified-random-sampling and nice temporal techniques like "sample-resample" where time is also a random factor in combination with geographical location as a random or defined set.

In outset all these techniques mentioned ( apart from sample-resample where you define the time period) rely on a knowledge of the size of the total set, maybe the national population or the geographical locations of all supermarkets. However, you can use educated estimates or other statistical techniques to produce a number.

Dangers and Benefits of Statistical Sampling
These techniques are reliant on a few assumptions, and often some manual intervention as to common-sense sample techniques. Here in lie two dangers though:

1) If you have say one model with assumptions for the population total and there are of course some assumptions in things like the t-test and Chi Squared test, then you can create massive sampling errors by the combination of the two levels of assumptive error.

2) You can become overly confident in the results based on a very good (and expensive) sampling methodology, while being completely let down by "non sampling error", usually a crappy questionnaire.

The big benefits are that you get a senisble and often very small sample size to then look into with GOOD study methodology ie you can put the man-hours into design, execution and results and not the shoe work. Also to the delight of many accountants or numbers guys with MBAs in senior management, you can present the margin of error expected from a sample size ( always " n ") and thus make cost-benefit decisions based on need for accuracy.

How Does this Apply to SMM?

Sampling is great for SMM because if your crawler is working to 99% or better, and your clients are only interested in the web sources you index, then you can spend time doing far better sentiment analysis, manually, than the algorythms will EVER manage to produce on larger sets of data.

You have two approaches in using sampling: sample the entire set of data BEFORE you index it or sample into results (or index-strata) for a given topic.

So for Twitter for example, with maybe 100million global tweets day to the minus one, it would be an efficient strategy to work on total per day ( you hear about it in the news, or you maybe can pay for the info, on a country basis too) and if you can geographically restrict your base, then you can work on the main number being in accessible languages and sample to a much smaller figure.

Comes with a Health Warning

Now here comes two dangers, the first one relates to the latter instance of twitter sampling: if you are looking to drill down to tweets on a brand name, in one country or language, over a short time, then you run the risk of NOT sampling enough records to give statistical robustness at this level of "cross tabulation" aka drill down. This is actually a big problem with many standard market research studies, because when you drill into say: Age by location by salary by single then suddenly your stats on that sub sample to compare them or predict them, evapourate into inprobabilities. Hence you stratify your sample: you use a common sense or pre-filtered method to include those sub populations or you study those sub populations against a sample of the General Public.

You see where I am going with reference to your itemised consumer posts being the "population", not the individual users.

Also the other problem is in this stratificaion: you choose a criteria which is not exclusive enough or exhaustive enough to satistfactorily capture that sample or that population to sample into. This would be the case with building a simple query, a search-taxonomy to dig out records in a certain topic area or a time frame, or geography. You don't capture accurately so all the stats-techniques in the world applied on that sample - population relationship won't save you from GIGO.

Beauty in Eating only Some of the Elephant

However the beauty of it there as a tool is also apparent: you can run very quick manual dips into either pre-indexed data (indexing takes time!) or into query results from your indexed data, or into your indexed data set as a whole. Then you have a back up too in delivering automated sentiment results and taxonomy query reporage: in other words you run the sentiment algorythm on total data for the period you want, and then sample into it and compare your manual ratings ( based on the same principles at least as the algortythm) done carefully on the sample. This gives you both a whole new strategy to add value to client reports; it also gives you a means of building taxonomy with a security you have enough threads to take key words out of; and it gives you a QA on your taxonomy or automated sentiment reportage.

DIY Social Media Sentiment Rating

Worse than that for an SMM company: it gives the average biology graduate, let alone stats' major, a tool to go and do it all manually by knowing forum sizes or tweet rates, and then looking up the random sampling tables to give them their sample points or periodicity.

It then comes down to how objective the human can be in allocating sentiment on say a five point scale to a post. Some people would prefer to leave this to an algorythm and I have sympathy for that!

Under the Bonnet of Social Media Monitor Tools

How does a Social Media Monitor Search Engine Work? Need it be so expensive to use when google is free? What can we do ourselves in house ? How do we cope with analysing 100 million tweets per day ?

Right now from what I have seen, the SMM companies combine several principal computing competancies (or modules if you like) all of which are actually undergrad' level, but of course require man hours and server resources beyond what universities will embellish upon "youff" or even post docs. This is probably why there are so many start ups in SMM strangling each other and dragging down the price chargeable for the most established consumer data mining companies!

Here are the key modules and processes that an SMM Search Tool will include:

1) The " 'Bot":
this is the robot or crawler/spider call it what you will which goes out and gathers information from the web sites. In effect it works just like your browser but instead ofg routing all the HTML and graphics to be displayed as a nice web page, it identifies the "posts" or "tweets" or other common text fields (or information like consumer assigned sentiment). This is also called screen scaping, but the complexities of forums and the perishability of tweets means it has to be more sophisticated, but still this is undergrad' project stuff.

Another issue here is that you want to probably have a link back to the original web page, you maybe want to track users by their name and time stamps for the posts, so your crawler-scaper has to handle this. As you get the drift, the eventual items will be stored in a database methodology and most often these days, in a MySQL back end database. The item gets its own ID and sits like a record with a few fields and an indexing methodology is applied so it can be found efficiently from keyword searches.

Now of course in the world there are lateral differences between web forums and of course FB versus digg or myspace, and these sites themselves vary longditundially ie in the course of their lifecyle they get re-formatted, revamped, extended and for forums often they get a new installtion of the "forum engine" or a completely new one. Now the project starts to take on the jobbing graduate (often drop outs actually who end up earning more than those who stayed to do their bachelors!) just to keep on top of the restructuring and when crawlers get kicked off forums ie the IP address or browser type is "sniffed" as being unwanted spying!!!

2) The "itemiser": This is the next step from screen scraping the information: it has to be repakaged in a standard format so that the firms software can use it ie all data should be standardised, and "Unitised". So another key difference to what your web browser and things like FF Scrap Book do is that the unit of selection is no longer the entire web page, but rather the individual snippit relating to one consumer comment which is stored and indexed.

3) The Indexer and Navigator: this is applied both to entire datasets and to individual entries. This is a little beyond the scope of this blogg and to be frank my know how outside XML, but all the words are indexed for each post ID : some indexers ignore common english words ("the" etc) , some run "triages" to index variations of difficult brand names ( eg acronyms, nicknames, abbreviations). Most SQL databases have an engine which will naievly index or can be programmed to the above, or you can build a specific indexing strategy in the engine or import the structure to it.

Indexing is in, navigation is "out" but in effect relies on good indexing. So you type in a search word and the system has a quick way of finding that word in the index and then finding all ( or in sequences of 100 say, per results page) the records which contain that word. This is the clever bit at google and yahoo: getting this to work at super speeds!

Computers are naturally top atheletes when it comes to running through lists and refering to "attached" or referenced data and then retrieving it. In effect Google and Yahoo make a library index card system, but instead of telling you the shelf location of one book by author or title from the alphabetical indexes, they know the shelf location by each word. There after they retrieve a short abstract ( the listing entry) and of course the link back to the original web page and display this for you ( at say 30 or 100 a time, the indexing has a ranking which ranks posts by various generic, pedantic or isoteric means and this is the soul of the arms-race in Search Engine Marketing /Optimisation)

3) The Web Portal : this is then, just a straight forward front end and "mid ware" to shuttle the query and results back and forth from the standard web browser, or the mobile device portal. You can also use the back end SQL interface itself, and sometimes an intranet http version is included so you can see how Oracle or MySQL think the web interface should work. Even a bolean search page and results service like Google Advanced or All-the-Web is now an undergrad' level project and loads of "home brew" programmers can do a pro' looking job in PHP. All this and the server ( computers at the SMM company end) are covered in other lectures.

Tuesday, March 22, 2011

Mini Me

Well here it is actuall, the mini CV

As I have said in earlier bloggs on job hunting, a few tips and experiences shared, this is a great way of working around recruitment consultants and getting your details kept at hand.

A CV is a private document so the two page A4 is both physically and psychically cumbersome. People want to file in a safe place, or shred vis a vis, delete and clear from Outlooks waste basket!

Consultants have had CD rom mini calling cards for years, and freelancers portfolios on the same media and I dare say a few punters have had their CVs there.

But now you have a wallet sized drop off!

I'd not put as much on it as the guy in the image, just something like " Mike Hunt, MSc Marketing 2011-12 ( ie not student in so thick ink) : Expertise In: marketing, inorganic c hemistry, last employer ( project) : dee dee dum. Contact, skype, e-mail, linked in.

Friday, March 18, 2011

Assumptions and the Black Box

One fundamental managment issue is what I call "assuming what the black box does" and this is taken in context of two negative outcomes: you either think the black-box process will perform as desired, or you think the black box process is completely unimportant.

Are you getting a handle on the manager's idea of the "black box"? In this monologue the said satanic box is clearly a process, product, service or system which delivers part of a larger process or system. Managers assume that it will perform as described or importantly, as they presume. It will deliver quality results, all on time, or on the other-sdie-of-the-coin, it is a current-operating system which requires no management attention.

There is a very topical case right now, and that is at the Fukushima power plant. A ten-fifteen meter wave and body of water could be anticipated perhaps. However everyone presumed they had adequate back up for failure in power to the cooling systems. The entire cooling system is no doubt "doubly redundant" in that there are two systems each with their own internal back up. I dare say they had many banks of doubled-up diesel generators in the event of power loss. However, they did not anticipate a swamping of these "fail safe" engine houses. They made an assumption, the black box, would deliver in whatever crisis may come.

Now I am a pedantic type when it concerns detail I am interested in and also "what if" scenarios. I should have probably become a safety engineer or the like, anyway instead I often hear alarm bells when I see managers either lighting up with the glee that "this black box will solve our problems!" or the condescending, eyes burried in paperwork " that black box is unimportant Jim".

Black boxes most often arise in corporations within either software, system management or consultancies, and wo-be-tide thee who combines all three on your desk as a general or marketing manager! The related group of nouns include: " alignment ; migration ; evolution ; deliverables ; and even basically Project"

Patenting as an Example of Intellectual Russian Dolls
One area I have been involved with, in some depth, is patenting. Or rather the results of "black box fixits" in the whole patenting process and contentions of patents thereafter. After a three year break from anything to do with the EPC and US Patent Office, it lands on my desk.

This is a very good case in point: a competitor was challenged for possible infringement. Very quickly however, the patent showed itself not to cover all eventualities or arrangements of the type of solution. It was a watertight patent, but only for one defined, narrow solution. The black box here was the patent itself.

The patent agents' and attornies' very own black box was how the Patent Offices would treat the patent and how it could be contended. That is the next process they pipe the application and "file wrapper" into. They DID NOT presume they could get an umbrella patent, in fact the reverse: they considered a safe way of feeding the black box with food it would most probably not spit out in their faces: a narrow, single application patent.

The agents then of course could call the patent "Patent of this type of thing" in the title, which is what managers read, and then tell only upon challenge them that an umbrella patent would never have been accepted, or been so expensive to contest that it is better with a nice defined little safe and most-of-all granted patent.

If you read line one of the "claims" then you know immediately that this is a simple patent based around the single design solution instead of trying to be an "umbrella patent" which covers ideally the whole concept. Management assumed paying 50,000 USD for the relevant global coverage WOULD deliver exlusitivity on the general solution.


So you kind of get the point...and moreover ....worse: there can be a russian-doll chain of black boxes. One could argue to basis of the western capitalist system is based just on that: you beleive the next black box in line to you will create value, and that is enough for your own little horizon. Hence the whole sub-prime pyramid sell could happen and completely undermine the entire system.

Buyer Beware Black Box Pendlers


I remember the film "total recall" where Arnie is sent an expert doctor to coax him out of his alleged dream world as a rebel on Mars. Arnie goes along with the story so far, until he notices that single bead of sweat is running down the good doctors brow. He then realises the trap and responds in true terminator style.

Now of course you can never know the exact workings of all those black boxes in the chain of systems or events around you. However, you can apply quality standards and systems like six sigma, to impose standards across all black boxes. Hence if six sigma is a commonly accepted and learned mantra, you worry less about what each box does. A different approach is to get expert advice, on an independent basis, or go and learn the gist of the system: this is where I stand. I either give-a-damn and learn as much as I can and feel I need in the system or I ask someone to show me it, or better yet an inpartial person to tell me about its stregnths, scope and weaknesses.

When dealing with consultants, sellers or internal managers, especially IT managers, you have to look for that bead of sweat, or that little bit of over confidence in their offer to solve the woes of the boss. Also avoid, as I have blogged before, ambushes in these situations, and rather make space for "scoping" and "scenario affirmation" or whatever BS, to basically smoke-out the BS and get to what is really being offered and what eventualities really may not be covered for. Then throw in a 10 meter high wall of ocean visiting you for an hour for good measure!

(c) Author 2011

Wednesday, March 16, 2011

Steely Scam

During the recession of the 1970s and early 1980s, there became alleged over capacity in steel and to a lesser extent, aluminium. Earlier in the 1970s there had been an energy crisis, which in part precipitated the recession ( citation 1,2 ). At the time of the recession, many steel works around the western world were either in public hands or subsidised and protectionism ( which is still an issue with countries like the USA and japan ( citation needed) ) was frequently used by countries trying to both defend their industry from low cost countries, and also secure enough supply ( and perhaps oversupply) for their key industries like car building.

In a very odd combination of command economics and free marketism, countries more or less conspired to drive out capacity from these industries, with the UK being a major loser in terms of jobs and then later loss of capacity.

So on one side subsidies should be reduced, and companies fully privatised while on the other supply would be "capped" so as to make the price of steel higher and thus more attractive to the stock market investors. As mentioned once again, many countries "cheated" and took legal loop hole clauses and "modernisation" exceptions to their actual plans.

While the UK under the Thatcher administration, went about taking out value adding processes such that smelting plants looked less economic and could be closed on grounds on unviability. So for example, the strip mill at Gartcosh which added value to the raw ingots produced at Ravenscraig, was closed by command.

Ravenscraig enjoyed economies of scale, local energy supplies and a deep water port in economic distance and should have been a prime candidate for privitisation towards a free market with its existing value-adding plants and local customers in especially the Oil industry.
However, the conservative government of the 1980s considered that the stock market would not want to invest in an industry with over capacity and potential for more competition from the far east at that time. Since they had few seats in Scotland, they chose this plant and it was not just Sir Ian McGregor's management board who decided on the action. ( citations needed). The same was also partly true in the UK aluminium industry.( citation).

A decade or so later, and steel was starting to come in short supply, with many of the far eastern smelters increasingly being tied to japanese, korean and latterly chinese customers where geographical cost savings and fortuities could be taken into account. Aluminium started to become an econonic alternative in some industries, and there too some shortfalls in supply drove prices up.( citations).

Dole or Subsidy in Transition to Market Metals?

The key socio-economic question is, was it better to subsidise and protect the industry as the USA did anyway through to the 2000s ( cite) , or to pay the social benefits and soft-capital to try and rebuild these communities' economies which were so dependent on steel and coal?

There is no real yes-no to this question because there may have been an inevitability that with over supply, and emerging low cost nations ( india included) the outcome would have meant that the UK industry was not competitive. On the other side, given a smoother de-subsidisation and privatisation with state as a beneveloent share and stake holder ( the current Chinese and Norwegian model for primary and heavy industries) and the up turn in the demand for steel which did happen anyway, then these plants could have been active in an eventual fully free-market ( in the EU at least).

Couple to this advances in technology both in production, logistics and then in the whole value adding chain, then the steel industry could have survived and thrived as a larger entity in the UK: whether or not the stock market would be particularily interested in the difficult overgangs period is another matter, but instituional investors may have been willing to both take the long term risk AND also contribute to protecting value ( eg savings and pensions, value chain share prices) in the economy as a whole.

Saturday, March 12, 2011

Dynamic Web Page Interface Languages

This is the first entry in the blogg version of lecture II in "web site languages". It is aimed at the non IT Manager as well as alma mater Strathclyde University MSc Marketing Students with no prior technical IT background. It is broken down in smaller bloggs for ease of reading and repurposing,


PHP is probably the leading dynamic language today

PHP has become probably the world leading language in terms of volume of web sites and transactions handled using this language. It is an open source language and engine so many developers cooperate in its development and it is available in free versions as a server engine. Modern browsers are optimised for PHP. As a code language it is one which is interpreted on the server by a "mother" programme, very much like the other languages discussed below, (asp, cfm).

Coding is both embedded in the html, in the URL constructs and in post-form data as well as in the files on the server side. It has evolved a diversity of really useful "routines" which you call up with a fairly simple pneumonic or english sounding programming command and are actioned by the server through its interpreter.

By in large, information from the client (you!) is requested in forms, buttons etc or refered to via cookies ( or if you are mid way through a transaction, the session file) and then inputted to programmed actions to then give planned results in information, web pages etc you get back. Thus what you actually see on the web page is often the result of quite a lot of dynamic computation, based on what the server had to work on in the second you visited or interacted. This can be in everything from building up a shopping cart full of goods and moving to check-out, to just the simpel dot com request for the home page.

Historical note on PHP
While Cold Fusion was ruling the cutting-egde-roost, PHP was the poor mans cousin: "Personal Home Pages" was a free shareware language which was used mainly in small web start ups and colleges. Microsoft then took the Cold Fustion crown with ASP, probably due to active-server-pages being better integrated to SQL Server language, which a lot of nerds were learning at university.

At that time, cold fusion programmers could charge a premium of even several thousand dollars a day! ASP was pretty quick to pick up if you had both HTML and SQL Server and Microsoft had the infrastructure to support wide spread training. Thus CF and .cfm lost the strangle hold they had on the market and macromedia faded into a buy out by Adobe. I actually moved a whole web agency over to asp from cold fusion because of both cost and the obstreporous nature of "cfm" programmers in 1999.

Meanwhile, the community of PHP developers were sneeking up and it wasn't long before the cross platform, open and cheap nature of using PHP and shareware MySQL erroded microsoft's heavier weight and contendably, restrictive APS_SQL Server structures. Being free and open source, it meant programmers could get mutual benefits in building upon the language itself, having peer review on the web for their prototype commands/routines, and then the whole language could evolve forward using the best types while rejecting the weaker ones.
I remember turning up at a fledgling web agency in 2000, having worked at an CF / ASP hot shop as a project manager, to see this home brew code in PHP and taking the whole thing as being amateur. How wrong we were all to be proven.

What Dynamic Languages Do and How

All these languages mentioned above, have a fairly common modus operandum ( but not perl script and some others very little used today) : when the server is configured to accept PHP requests and run PHP commands server side from those client requests, then input strings/ forms / requests go to the appropriate area on the server set up and allow the programme to execute a response, and most often send out information back the user on the internet. Most often this will execute a database query based upon an input string. Otherwise it could find content from a source which varies: a file location or third party web site: by just changing the one parameter, like the contents of the folder or the web address, you then serve up new information without needing to.

Information sent to and from the server can be text based, mediated in various string forms or XML, or it can be graphic or other "sub files" to be used on the web pages at the client end, or it can be array based where data is hidden on the client end and presented upon request.

These programmes like PHP and their eventual coding allow for several useful operations to be combined in one command, which has often some "english" comprehensibilyt eg $_GET is a way of handling the strings from GET

In php you can choose to use GET or POST when forms are submitted : post goes behind the scenes so to speak, while GET places the search string in the URL
You can look into the "under the bonnet" workings of many web sites if there is a GET command in the URL when you interact with the website. Usually this approach is submitting a query behind the file name

eg. www.xyz.com/page.asp?ring=3&type=30&date-01-01-2011:01.03.2011.....

The question mark denotes the GET , the first or primary element is defined and thereadter ampers AND '&' is used to both define the operators, the conditions and any bolean AND search terms which are either included by your typing in a word or as part of your order processing for example.

If you come across one such GET line in the URL then you can play around with building your own queries and submitting them to see what works and what comes back. As a web developer you can also of course then use this web site to deliver queried information via your own server ( but not on web pages due to the 'same origin policy' enfored in browsers, see 'security'). The only current alternative to scraping content or 'proxy' serving queries is to use RSS feeds from 3rd party web sites and then parse their content into the web page or create an on-the-fly array to query.

Another way of analysing dynamic web sites is to use "firebug" and right click on any dynamic element and "Inspect Element" which takes you to the code directly. This is invaluable because web code can take ages to read through and it is easy to miss a table element for example. This is one use of the term "screen scraping" in this case for dynamic code and not XML content.

> ie www.searchmonkey.com/search.php?q=monkeys : the apache gets the URL request and knows to both go to the search.php location and send the string "monkeys" to be processed by the php there.

The php programme must be installed to the servers network: CFM used to just be a disc or an online download for ISP web hosts. The primary end of the server network, usually apache, then sends requests for php pages from the browser to the appropriate file location.

PHP has some clumsy language and syntax according to many programmers, but it is relatively client-server light and easy to learn. PHP v 5.1 and 5.2 where stable and debugged largely in 2010 for all browsers and most server types.

There is a master command file, an initiate / config file usuallycalled " Php.ini" which sets up what elements and syntax will be allowed.

Despite the criticisms, PHP above all delivers reliable and scalable web sites. Programming resources are now widely available, with the ease of learning and share ware nature rendering dynamic web sites within the reach of many amateurs. It is very much a language that is here to stay and will continue to evolve with the browsers and server technology. So for the forseeable future we will see many web sites or at least pages, based around PHP or wholly dependent upon this dynamic language.

Web Site Languages Part I

Languages Utilised in Internet Technologies

This is part of a larger lecture-blogg series which is aimed at marketing students at the University of Strathclyde, and non IT managers. In this lecture, and the following, we introduce some more terminology and a behind the scenes look at how modern web sites are programmed and how they interact with you the "client".

Introduction:

How and Where Does All This Fancy Stuff Get Done?

We have considered the simple protocol languages which enable communication to be ordered and policed on the physical internet. Once connected to a server, in order to present and interact with web sites, there are computational tasks mediated both by programmes on the web server itself, like Apache and PHP, and on your own home computer, mediated via your web browser programme.

Both "server side" and "client side" language-engines rely on reading instructions ( code) from files which are either downloaded or refered to, called upon if you like, to perform tasks in presenting the web pages, shutteling information and performing calculations.

For many programmers, the goal is to have most of the work done server side, such that communication is quick to and from the client ( ie you and your IE Explorer/ FireFox). This approach is known as the "thin client" and has become popular with the growth of mobile phones being used on variable connections. This approach limits the number of http requests and the size of code in each download to the client machine, thus reducing front end server load and outward bandwidth requirements.

The downside of "thin client" is that with more complex web sites and interactions, there is a heavy computational load on the part of the server structure, usually a second layer of computers, which are then required to work more processes. This requires more thought in "vertical scaling", a discussion taken up in another lecture-blogg. Also, although the processing reduces code download, it may in some cases increase the amount of raw data because little can be "expanded upon" on the client end.

However, to provide the most rapid web experience, a compromise is often reached for the more data-rich web sites, like Google maps, whereby the local client runs an API ( see below) and requests packets of information without needing to reload the entire web page. ( more detail on this in the second lecture in this series). Many mobile phones have specific "apps" for web mediated services like mail and social media sites, which offer a slightly thicker client at a single download, but are better optmised for speed and screen presentation for the small screen environment.

In some more technical depth:
In essence most languages used in building and running web sites, which run either client- or server- side are actually instructing a mother programme to perform computational tasks: that is to say the language coding itself is not interpreted or compiled directly.

Exceptions to this would perhaps include running Java modules on your local 'client' PC, some PERL script used in communications, or using "C" and other interpreted/autocompiled languages running server side from instructions sent forward from other languages like XHTML, PHP or Javascript.

At the client end, ie you, the mother programme is most often the internet browser ( apart from Java, which is presented through the browser window though) , and some of the updates you will receive to browsers are important because they allow for the latest advances in the languages to operate on your machine. For example the "javascript engine".

Server side, it is really dependent on what you install: php , cfm etc all come with an installation disc and these "interpreter" programmes are really very large relative to say the javscript element in a browser.

HTML


HyperText Mark-up Language is the standard language type for web sites, but it does vary and evolve so that the earlier web browsers like IE version 5 for example, do not work with more modern web sites. Also some browsers do not automatically fix bugs in the HTML code.

In essence, HTML does as it says, it allows for text to be formatted in a simple, high level ( ie "near english instructions" in the code) language which is easy to learn for anyone slightly interested in computers with the motivation to make web sites! Apart from formating text into headings, fonts, paragraphs etc, the code "coralls" the structure of the web page and how it expands etc.

HTML only web sites are very outdated today, but the language is still at the core of structuring web pages and allowing for communicaiont. Indeed in its later forms, it coralls both this stucture, communication to-and-from the browser, and also the inclusion of elements which utilise other languages or call upon other data sources.

It is VERY worthwhile as a marketing student to take a course on HTML and learn how to read the "anatomy" of web pages using tools like Firebug. Even if you never actually build an entire web site yourself, it will allow you to edit things like links or titles quickly or paste up emergency notices, or take out elements immediately for legal reasons. We go into little detail here because it is a subject in its own right.

So HTML still forms, for now, the core of most all web sites, but other elements mediated by other coding in the languages below, and the use of API elements, are reducing the volume of HTML code to be found in a modern web site. Hence we take up the thread with the most closely related language to HTML, XML:

XML

Historically, XML's roots predate HTML in it being related to a form of sharing ASCII/ Unicode based documents across formats. This goes back to the 1970s, and the forerunner "standard-general text Mark up Language" : SGML. XML takes this cross platform, simple form of sharable text mark up to a form which is inclusive of many desirable features and flexibilites which now mean it is used for both static text and very dynamic data-shutteling.

XML is very like HTML: it is near to being easily interpreted in english, ie high level, and in that it marks up text with mark up tag and then allows for content to be basic Unicode text. There are elementary mark ups as in HTML which are simple and allow for documents to be parsed into other web sites and presented in different formats based on the key heading and paragraph strcuture. In fact XML is to a large extent, a simplified text handling approach than HTML, or cleaned up because HTML has evolved to have many other functions while its text functions have become somewhat limited.

Features of XML

Mark Up : Text Document Standard, Shareable Formatting:


Presentation: Integrates / interoperates across platforms and repurposed documents like pdf output, or .doc/.xdoc. Apart from text, there are also there are some vector based graphic elements like simple flash which can be useful.


Exchange: used to shuttle data in HTML, javascript/JSON/AJax and also between non internet browser systems: like from different ERP or legacy systems, or from web site orders over to ERP systems by an indirect system ( not a true "Back End" database set up )

Programmability : Eg X-Query ( SQL liknende for XML) , X-Path routing/indexing tools to search data/operate data in the larger XML file(s) Simple API Xml integration SAX.
XML celebrated 10 years in 2009, and has come into extrememly frequent use in web sites and shared news or document sources. XML documents.


XML has become "interoperable": because of its simplicity, other programmes apart from HTML engines (now XHTML) in browsers can integrate quickly to the language and extract information from XML files readily. In terms of the language, it is 'neutral' in that data can be exchanged between languages and operating systems / platforms. Some XML formats are very data centric: The benefit over 'flat' csv/psv/tab.sv files is that the information is "marked up" with syntax/formatting tags or other tags, and also custom field name-tags eg"
........" can be applied to text strings, thus rendering XML a database resource.


One benefit is that the structure of files is easily readable, in fact a pure XML file is usually easier to understand than a corresponding HTML. Also, older ASCII text sources or scan-read sources can be readily parsed into XML and then it is easily shared to different web sites with their own formats.

Mark up tags can be programmer definable, which is one area of true advanced capability with XML. The second area, often lumped into the umbrella term "XML" is using APIs and server side app'lets to access the data held in XML repositories: often this is like a simplified SQL.


The evolution of XML continued until a point when the drawbacks of all the embedded text in HTML became so large that the simpler root to text data was attractive enough. In other words, the "atmosphere" around text embedded was suffocating the flow of information on the internet and necessitating re-purposing of text between web sites and systems.


With the widespread use of Cascading Style Sheets, XML became even more attractive, as the one source of information can be repurposed to different styles automatically or in a browser type/ version dependent way after sniffing the browser.

Yahoo YUI has good javascript applets, like a pop out calendar.

Another benefit of using XML files and information sources is in serving the same web site to different devices or bandwidths: so for example it is still the case that javascript on many mobile devices ( phones) is either very limited or not there: therefore you can have a simpler web site with the same source in XML for text information: the same is true in optimising screen layout for the window in iPhone, Windows Mobile or Win CE

What this all means is that text documents can be published once in simple XML and then accessed from all over the internet and republished in the format of the web site requesting the XML from the link or serveer side source file. When using APIs, this means that the document content can be accessed and presented in more dynamic ways by using small programmes rather than having to place the XML document or text in a back-end server-database system with perhaps three layers of access: apache web server; MY SQL interface ; Routing interface and data repository. Also this allows the programmer to develop very defined small client side javascrip applets which will have very specific, targeted functionality: for example, find the latest news items relating to the US Senate from three XML sources ( RSS is an XML subscription app')

XML is fast and simple for text based dynamic web sites with several internal and external sources of updated text.

Also another very common use of XML today is in form submission and handeling of orders on e-commerce sites, especially when there are "Under vendors" ie suppliers external to the main web site. Using XML the information can be passed in a simple and cross platform message, compiled during the session. Here XML is an intermediate, common format which allows for specific tags useful eg ...

In fact any UID database which can be output toa tab/pipe seperated file can be parsed into XML and reverse parsers for ORACLE, SQL Server etc exist: field names move into the tags
for example and then the data can be imported intelligently to many different systems, with manual reference made somewhat idiotproof by the use of english and a simple mark up coding.

It need not be documents, any ASCII Unicode file can be used, so for example a tab or Pipe separated database could be encoded and referred to by simple searches and operations server side, with the form going as XML.

One powerful capability with XML is being able to link different sources of data with a unique ID : for example you can combine some graphic like maps with post codes or coordinates (longditude/latitude) , with a text source like population. This is used in the tiles for google map and satellite images for example, and to link the simple geotags for the icons which appear with a pop out text box. This is often done in APIs using the standart API Xml module SAX.
Structure in XML files: XML files should take the schmatic that the structure of the text is like a tree: there is a sequence of nesting data within its tags, or branching out if you like.


So these form a simpler version of tables in a database, and you should consider structing different XML files with common, strict UIDs so as to be tidy and offer more functionality, while keeping individual file size small and closely clustered in relevance of data. XPATH helps navigate this efficiently. So XPATH may be used to create an efficient, exclusive search which goes to
and returns for example.

You can ad alot of tag attributes in a text document you want to be highly structured/relational/ nested in searching, to give exclusive search results or actually just serve up the relevant information in terms of pages, paragraphs or specific hit count with links to the lines of text ( the actual scentences) : so you tag up "US presidents" beside each relevant name and maybe add a period-of-office date range whereby you can find each president, who was when, and which of them reigned over a period you are interested in, without knowing the term of office dates themeselves.

ISO8601 is an ISO standard on date-formats contained within data fields, which is adopted into XML easily and you can find it easily. The same is true for long-latitude Geotags.

RSS ( really simple subscription)
RSS is a subscription news feed service which works through XML and both allows you to get updates on news or alerts, view them in any browser and then also re-publish the xml: so the example on the Harvard E75 course is using a javascript interface to show both google maps and "geo tagged" news feeds as links in pop up bubbles upon rolling over town locations. RSS can also be used for other not text based updates or as a quick way to move (push!) small amounts of information to a web site a user is subscribed on: so for example Pod casts which should not really be included in RSS by the spirit of the original convention, ie this pushes a file and not jsut a simple news item.

Podcasts on iTunes/ other clients are actually based on RSS with a media file linked in.
A web source will publish RSS to a URL and your insertion to your web page or server side parsing.

Now RSS is useful too for capturing information form syndicated/external sources because it is so simple and allows you to parse, then search, store, reformat and summarise/ abbreviate information from a web site which otherwise is not doing XML code you can integrate to ie parse in the whole page and dig out the XML marked up text you want, called actually "page scraping".




XHTML : this is the most XML friendly version of HTML, which has some other syntax and case sensitivity to XML tags, but all later version browsers run it because it is XML optimised. XHTML also allows for otherwise illegal characters,

XPATH
is a sub-language which allows for SQL style querying of such data. Thus you can have either XML on the page, or hidden, as either permanent dataset or something dynamic, even transient, and query it without the need for a complex back end database interaction at this point.

AJAX was a synthesis of javascript and xml, which allowed for client side javascripting to request required data as simple XML /XHMTL, rather than having that intelligent computation on client behaviour, happening by http to the server and back. The best known example of this is google maps, which was the first to run a credible javascript scrolling system for showing graphic and bitmap link. The javascript detects the scrolling locally and asks only for the packets of information you require, and incorporates them without any more server intensive processes needed. The term "ajax" is now used to include XHTML and JSON as well, we will return to this in the next lecture.

API: application programming interface:
a function, or a little premade applications, which is available for your to call up and utilise in your web site. This differs somewhat from a "web service" which is a web server which performs a service for you on their processing time: eg tweet decks, so you use a GET$ URL perhaps to send in the data or request for computation to happen there. An API works in javascript on the client side, and these are shared as programmes to allow web sites to propagate their service externally while most likely holding the upidated data and say related push advertising from their source: eg Google maps, Twitter, Facebook linking/write to FB. We will also return to APIs, like google maps in the next blogg.

User definable tags means that whole new mark up languages can be made within XML: for example DNA sequence mark up language.


The next lecture will focus on the leading dynamic languages, starting with PHP and moving on to the most modern, Ajax/Json which keep Facebook and Google Maps hyper-dynamic!