boss – THE MIGHTY DATA

4 TYPES of BOSSES WHO DO NOT UNDERSTAND analytics

When I wrote a blog about the Analyst Loneliness Syndrome a few weeks ago, I knew I wasn’t talking about isolated cases. However, the magnitude of readers’ responses have completely knocked me out. It is bitter-sweet mixed feeling of sadness and joy that you nailed something right, but you feel sorry that so many people are suffering from this syndrome. So I decided to talk to some of those who contacted me about that blog and to write extension of original post. This time about one of the three core factors of analytical loneliness: the management side of things.

The classic HR maxim says: “Employees don’t leave companies, they runaway from their superiors.” Although I do not quite agree with this generalization, I have to admit that in 4 out of 5 cases when I changed my job, it was true (this is my greeting, Rasto, you are the exception). One can leave boss behind for a variety of reasons, but in most cases it’s a combination of some of the following “evergreen’s“: He can’t appreciate my work; He does not understand the area and therefore I get mostly nonsense tasks; He does not believe me and hints me so; No inspiration or development from him, I only rot; His/her moral standards and deeds are in deep contrast with my beliefs.

Certainly, managerial superficiality and incompetence can affect you in almost every industry, but I would like to specialize in a typical example of this ailment in Data Analytics and Data Science. The traditional managerial characters have gained some additional spicy ingredients in this industry. After all, judge for yourself, here are 4 TYPES of MANAGERS that don’t understand analytics:

1] Don’t drag me into details

Managerial Profile: It’s incredible, but even today there are still many companies where data analysts is “stuck” under the Head of Business or Marketing. It is often a tragic consequence of widely spread belief that data can significantly influence company’s revenue. As a result, analysts are moved under the Head of business or marketing to make this influence happen . However, these are usually managers who have a mathematics aversion, developed already at their (primary or secondary) school. Anything more complicated than percentages leave them restless. Simple numbers like sum and average (of course, they only mean arithmetic average) ok, but everything else is already far too complicated. Any statistics beyond the correlation are just “academic curls” (or crap). Their phobia from numbers and more sophisticated analyses comes from the fact that they have never understood this area, are not in control of it and thus are afraid of it. They don’t believe in the power of calculations nor AI, they solve everything intuitively and on the basis of proven approaches (read as: it worked once in the past). He prefers human speech as communication, simplifies every schema or spreadsheet into 2-3 sentences. However complicated the analysis is, at the end everything has to end up in Excel, which can be filtered by columns, and must not be more than 50 lines. When you try to “dive into” the results of your work, (s)he will tell you “Let’s do not get too technical” (just tell me the essence)

Implications for your work: If you work for such a manager, you will be probably having a very frustrating working life. Since this kind of managers never did any analytical work, (s)he doesn’t know what IS real and what’s NOT. Neither in terms of procedures and results, but especially not in terms of time you need. So get ready to receive ridiculous tasks in gallows deadlines (what could take so long, right?). He has intuitive expectations about every assignment he gives you. If you fail to match it with real data, a tough week is waiting for you. No matter how thoroughly you prepare your analysis output, (s)he will take one or two of the most obvious (= most primitive) conclusions, thereby gradually discouraging you from coming with more sophisticated procedures first place. Sooner or later, there will also come attempts to censor “illogical” analysis outcomes. If it is necessary to present to “seniors” conclusions, (s)he will let you do it (while throw half of the slides out of deck as useless). Because if the top management did not like it by chance, (s)he will drown you with “it not making sense even to him/her”, but it just happen to be the calculation result. In the area of expert or personal development, you are down to pure fate of Robinson Cruse.

What should you do about it: I hate to be an evil prophet, but if you are serious about your analyst career, run away from there. In fact, this kind of manager is unrealistic to improve, because he considers more complicated analytics to be a necessary evil that suits him only when it confirms his intuitive hypotheses. Otherwise it is unnecessary “trying to look smart” that has no support in (his/her) reality. The only alternative to fleeing would be to attempt a coup d’état (whistle-blow him to a higher level of control and they might exchange him). But honestly, this kind of managers have a stiffer root and they have more “merits” than you have of convincing arguments. So sooner or later you just leave (with great relief).

2] Scared rabbit

Managerial Profile: This type of manager stems from the first type and often represents a generational shift or personality development from “Don’t-drag-me-into- details” type. What remains the same, (s)he never did an analytical work himself, so things are not understood. The “move forward”, however, is that they do not reject a more sophisticated analysis because it has come upon them that they cannot do without it anymore. To this “improvement” he was pushed most likely by the CEO / shareholder attitude or the fact that he noticed all the competitors around already using analytics, so we must have it, too. However, as (s)he does not understand things him/herself, he is only trying to follow very elementary steps, often mimicked from professional conferences or buzzwords (anybody Big Data?). (S)he is stiff whenever you enter her/his office, because (s)he knows that the debate with you will revolve around an important subject (s)he doesn’t control. Nevertheless, in order to survive (s)he must feed to levels about him/her (who forced the analytics first place) the illusion that (s)he is not only interested in analytics but also orientates well in it.

Implications for your work: The consequences are similar to situation when you have to get out of a dark room filled with things. (S)he only progresses slowly through the familiar outlines, (s)he first gropes everything thoroughly to make sure we don’t bump into something hard.As a result you will only get elementary assignments, everything will have to be tested in a small pilot (= no effect anyone could notice). Concept that to you train a model first on 1,000 people and then scale to 100,000 doesn’t make sense, does not ring a bell with her/him. Therefore, most projects will die after the pilot. He’ll never fight for better software or a more powerful computing engine, “let’s try first with what we have. When we do, then we can ask for more money.” (S)he’s too soft, because (s)he can’t steer you by essence (since (s)he doesn’t understand it) and so (s)he will try to do it in a moderate way. You won’t get strong decisions or quality feedback from them. Do not expect a vision where to follow, you often will be firefighters of issues that fell from top (and which (s)he cannot conceptualize and prioritize). Since (s)he is uncertain in your area, he will explain everything from Adam (sometimes repeatedly, as it has been overwritten by other issue in his/her head). Most probably he will never let you present the results of your work, so that it is not revealed to leadership that he does not understand even half of what you do.

What should you do about it: If you do not mind (or even prefer) that this kind of managers isolate you from contact with the top management, you can survive in this setting quite comfortably. However, you will have to educate your direct superior continuously (sometimes repeatedly on the same topics). Do not expect any career growth or expert development, at most you will be left with the space to self-tune. As a intermediary station, this is not a completely unbearable. But primitive and repetitive tasks and professional stagnation will catch you up sooner or later. If you have lived with such a manager for more than 3 years, look around where your peers have moved. Your train might be running away.

3] When we in ’95 did this …

Managerial Profile: It is a manager who once worked as an analyst. Of course, when data analysis meant OLAP and mainly SQL data reporting. (S)he didn’t get too wild with predictive models, Monte Carlo simulations, or neural networks. So (s)he did not realize that data analytics is done completely differently today. In addition, his/her abilities are more of a memory-optimism that is often transformed into “When we tried this way in ’95, it worked”. In a sense, this type of manager is more dangerous than the first 2 named types. If some tries to convince you of something that is true, it is always worse when (s)he thinks being right rather than being not sure about the issue. In addition, this type of manager wants to be involved in every detail because he remembers that it was exciting to reveal new connections (maybe (s)he is nostalgic about it even). In fact, (s)he does not realize that “is no longer playing the same league as the young ones”.

Implications for your work: Perhaps the biggest risk of this type of managers is micromanagement. By living in belief that they understand the area and by having nostalgic memories of the times when they did something real with the data, they will seize every opportunity to “engage in the project.” This can sometimes go so as far as to “volunteer to help” and take parts of the project on their shoulders. (what is to be avoided by far, if for nothing else at least to meet the project deadline). Speaking of those deadlines, the second major risk of working with such a manager is unrealistically optimistic time-frames. After all, when we did it in the 95s, it took just … The biggest risk in the long run is that it will slow down (or “torpedo” by expert “arguments”) your introduction of the modern trends (to keep up with you). Maybe (s)he won’t even do it consciously, but if you take two steps back, after a few years you’ll find that you are more or less spinning in a vicious circle.

What should you do about it: For some people, such a job can be comfortable and they let themselves to be fooled that it might have turned out much worse off (see the first and second type of manager). If you are at the end of a career or are among those who prefer traditional to innovative, just enjoy a comfortable life there. However, if most of your working life is still ahead of you, you need to foster space for professional growth. And the pace should at least match the market growth to avoid becoming “unnecessary junk in the labor market”. Therefore, I recommend that you sit down with such a manager and ask for autonomy: part of your working time (e.g. 1 day a week) to test new trends (which (s)he does not hints you to). If the manager does not agree, (s)he is probably well on his/her way to transform into Type 1], and so should your answer to it be in the spirit of advice for that type (see above).

4] Jules Verne

Managerial Profile: To avoid wrong impression that a manager is a problem only when (s)he knows less about the issue than you do, there is also the opposite case. I personally hate the principle, when the best surgeon is nominated to be the hospital director, with the argument that others appreciate and respect him. Regrettably, even in analytics, the most skillful (or the most powerful) analyst becomes a team leader or department manager. It happens so often because the levels of control over it are some of the first three types, and so they need someone to cover the technical side of things. Jules Verne is a type of manager who once was Data Scientist or at least a sophisticated data miner. After (s)he stops officially being responsible for direct performance, and is charged with task to manage other analysts, often one of the following things usually happens: 1) Becomes lazy and realized that (s)he no longer wants to return to writing queries or code (resulting in a gradual loss of touch for analyst’s work) or 2) will finally take the chance to do those cool types of analysis that the nobility did not allow him to do before. Often both of these transform into a non-critical acceptance of “hype news” in the industry. After all, he also wants to brag on the beer with other data managers what cool things we are in our company. As the consequence the journey becomes a goal. Trying this-or-that is more important than making something easier to really work. While (s)he is no more responsible for time spent on the individual steps, rather (s)he already determines the strategy for future.

Implications for your work: The assignments become increasingly confusing, because “Try to plug in a neuron net and let’s see what it brings.” Of course, half-successes go into drawer immediately to free up the runway for yet another new approaches to try. The result is a frequent change of priority and a gradual absence of a sense of real effect. The absence of value added gets noticed soon also by the “those up”, as will the time pass working in the Jules Verne’s team also means an increased risk that some organizational change will wipe out the entire team from Earth’s surface (read org chart) without any warning. At the same time, this kind of managers push their people into the position of generalists rather than specialists, which must not necessarily suit everyone well. Projects’ track record might look impressive in CV, but when you gonna by interviewed by someone who really did that (and not just tried as your team did), you will badly grill on your own barbecue stick.

What should you do about it: If you are JUNIOR in this area, it is paradoxically more advantageous for you to stay for a few years. Getting a broad (and shallow) outlook at the beginning of a career is not necessarily a bad choice. However, do not take too high a mortgage so that you do not bleed when your team suddenly ceases to exist one nice morning. If you are a SENIOR, confront the manager with the flicker that (s)he shows. Give him/her a feedback that you want to finalize the projects and that one new idea a week a probably enough. If he doesn’t understand or laugh at you, go to his supervisor to describe the situation and say either HIM/HER (or YOU). Both answers will be the right choice for you. If you are the first to do this, you probably save the rest of the team, but you will not regret the possible departure (possibly with handsome severance pay to get rid of you quickly).

Have you stumbled across one of these 4 types in the workplace? Have you ever experienced yet another type of dysfunctional Data Manager? Share your impressions at info@mocnedata.sk. I keep my fingers crossed for you to avoid those types of people. And if you happen to meet them, try to follow the advice from this blog. Bon voyage!

Späť na domovskú stránku

Elections are like Hadoop

In my mother tongue, when one does not know/want to share reason for his actions he says “Lebo medved” (= due to the bear). That brings the actual bears off balance, because they also do not have a clue why you would do that. Going through several elections in past few months, I realize that elections are “Lebo Hadoop” = a bit like Hadoop. Don’t you get it why? Well, then let both you and yellow elephants understand why:

Hadoop and other technologies of distributed computing and data analytics are still less common in our society than the progress would expect them to be. Upon explaining the concept to managers I quite commonly have to reason that implementing Hadoop (which they heard of at any business conference lately) is not switching Fords for Volkswagens or like migration from MS SQL to Oracle. To really embrace the distributed technologies is rather massive change to company business processes. As this stage I usually run short of managerial understanding and start to hear statements like “After all, Hadoop is piece of software, so it can be reinstalled to take over the actual SW, isn’t it?” You roll your eyes and start to think if it is actually worth investing further clarification effort, when the chances to succeed are so small. And exactly for these situations comes handy following explanation:

In most of the European countries elections are organized through means of voting districts. Every voter is registered to exactly one polling station, usually the closest to his/her living address. That helps to make sure that the voter does not have to travel too much to polling station and thus allowing to minimize the time needed for single voter to cast his/her vote. Immediately as the election time is over, the local polling station (through hands of the local electoral committee) starts to count the votes casted for different candidates. After they finish they work, they create election protocol summarizing all achieved results of vote in given polling station. This protocol than travels to county electoral committee to sum up all of their polling stations and then through means of regional and national electoral committee to sum the entire election result. We feel this process to be, somewhat, natural, as we got used to that over the decades. But let’s imagine it would be all done differently …

Imagine the whole country would need to vote at the very same place. That would mean some people will travel very long distance (what certainly would influence participation in elections). What is more, imagine how big would have to the polling station to be, to actually host that millions of voters. And how long the queue would be to actually cast the vote. Polling station that big would have to be a dedicated building built just for that purpose (that is of no other use outside the elections). This way organized elections would be only possible with flawless voter register. Any small discrepancies in recognizing or registering the voter would clutter the queue waiting to give their vote. This election would be ultimate hell for local electoral committee. They would need, literally, weeks if not months before single committee counts and verifies several millions of voting ballots. And imagine that some of the electoral committee members get ill or go on strike. As there is only one electoral committee in whole country there is no replacement at hand. By this moment, you are probably getting irritated and think “Who on Earth would go such a down right insanity?!“

Well, you might be surprised to find out this is exactly the way how our legacy relational SQL databases operate. They try to store all data into the same table(s). As a result, the computer that can handle that much information, has to possess immense capacity that is both (as single polling station in country) too robust and too expensive. What is more, these cannot be the regular PCs honed by normal users, these have to be dedicated machines (usually not utilized outside of database tasks). All data points have to be written sequentially what automatically creates long queues if data load is massive. God save you, if single line of the intended data input is incorrect. Writting process is then aborted and the rest of the data still waiting to be injected. If the main database corrupts or some calculation got stuck, the whole business process freezes. Any slightly more complicated count takes enormous time. As more and more data flow in the process, it only gets worse at the time.

After seeing all this, Hadoop has said: Enough! It’s operation and data storage are similar to way we organize our elections. In Hadoop the total data is split into great number of smaller chunks (polling stations) that hosts limited sub-group of (voter) records. Data are stored close by their origin (same way the polling station is close to your place) and thus the reading and writing the data is much faster. When the (election result) sum is called, several (polling) places work in parallel to count the votes, not single election committee calculating everything. After calculation is completed on individual places (Hadoop calls then “nodes”), the nodes pass the result to higher level of aggregation, where results from nodes is summed to total result. While the node are calculating rather small and simple amounts, common PCs can serve as host for the nodes, no single supercomputer (one mega election station) is needed. These common PCs can have other use outside the Hadoop calculation (same way polling stations turn back to schools and community centers after elections are over). The analogy is real, as the Hadoop nodes have similar autonomy as the local electoral committees have in voting process. Same resources available to node are both steering the read/write process as they are taking part in calculations (similarly to local election committee members both casting their own vote as well as taking care of counting all votes casted). And finally if one node (local polling station) fails, another neighbouring node can take over the task. This way, the system is much more resilient to failure or overload. (imagine it to be the case of all voters have portal electoral ID allowing them to vote in another polling station if they home one went on fire)

The Hadoop geeks can object that the Hadoop-Elections analogy is not exact in two points: In order to prevent data loss, Hadoop intentionally stores the copies of the same data chunk to more than one node. If this was to be replicated in the voting process, people would have to cast the same vote into several polling stations “just in case one of them goes on fire”, which is not really wanted phenomena of proper elections (leaving aside the fact that there has been one legendary case of this actually happening in Slovakia). Secondly, there is not fixed, appointed state election committee overseeing the work of the local election committees. In Hadoop any local election committee can take role of the national one. However, leaving these two tiny details aside, the Hadoop-Election analogy fits perfectly.

So next time you need to explain to your boss (or somebody else) how Hadoop really works, remind them of elections. Or simply say “Lebo Medved”…

Späť na domovskú stránku

LONELINESS of data analyst

Higher salary. New challenges. Misfit with recent boss. Desire to get hand on much larger sets. Or pure world peace. None of the above is the main reason for data analysts to change their job. But what is then?

Not just American beauty contests, but also job interviews have quotes like “I wish a world-wide peace”. These are ridiculous statements on why you decided to change the job. I interviewed more than 350 people in my career and thus there is no shortage of even bizarre statements like “My father greats you, he thinks Allianz is still very good company and hopes you gonna get me the job.” Since 2017 I have been doing the interviews in a new country. I thought I would be learning some new “world peace” equivalents of this market. However, my surprise has been much bigger.

Loneliness as the denominator

I felt sorry for the guy, when it appeared in first interview. But when second, third and then forth candidate has come up with same reason of leaving recent job, I got on my toes. I thought if this market is some kind of fairy-tale kingdom cursed by mighty wizard. I could not comprehend, why so many people have loneliness as the common denominator of their life. This was not relationship- or friend- kind of loneliness. Rather this was all about work, the analytical loneliness.

How the story goes …

You graduated from some technical or IT-savvy economic university. You love puzzles and quizzes, you loved Math competitions and finding new interesting links between things. Thus, you look around the job market, where would you be able to put your talent into use. You attend several interviews and find out, that your potential employer has serious data conundrums ahead and that analysts would be measured against demanding criteria. You are thrilled. This is exactly what you are longing for: Let some task beat the f**k of you, so you can learn something new.

The first weeks on job is really fun, indeed. You discover new data that nobody analyzed before you. (even after you, but this you will find only later on). Demanding analytical tasks turn out to be a rather trivial data queries, often falling into reporting only category. But work is fun, so never mind. Your boss does not really understand the nitty-gritty of your work, he would not be up to finding you mistake in your queries. But he is really nice. Time flows, you achieved your quarter, year one, two … oh, man, how the time flies.

To your second Christmas in the company you beg Santa for a new boss, new project or at least the new version of the software you are working for. Just to get some progress ahead in your work life. You try to switch the job, try Start-up, Large corporations, family-driven business… Always few moments of thrill and then Yo-Yo effect of frustration hit as with diet. In the turmoil of internal fight, you sign up for expert conference. After all, one can get inspired externally as well. You meet a lot of stand-ins with same spark fading in the eye.

Loneliness of data analysts

With a bit of the alteration this was the common plot of all 4 candidate stories. None of them asked for salary of company benefits. None of them was investigating the chances of career growth in future. All they were interested to hear was WHAT and UNDER WHO’S LEADERSHIP will I work on? The usual comment went like “In my recent job I am the only one doing data science. Nobody understands that matter, I have no one to consult my approaches. I only know what I googled out on stackoverflow.com or similar portals. I feel lonely!”

It is interesting. When I gave it a deep thought, I realized I can find stories like that in my friend circles, too. The job of analyst is now going through strange wave. There is enormous demand for data-analyzing people. However, since Data Science is very young branch of analytics, there is very few managers, who actually did the Data Science themselves. Analysts often report to managers, you are not experts in the area. Thus, the young data scientists are doomed for the path of the bonsai: Nobody expects you to grow big, they water you time-to-time, but you are just cartoon of the real tree. The limitations of the expert growth of analytics (subject to another blog coming soon) are dire. These candidate stories only remind me of that again.

The same chorus 3 times again

Now it is clear to me that these were not isolated outliers. Sadly, the analysts’ loneliness is indeed a bitter interplay of three factors. Firstly, there is still very few data scientists. Therefore, it is unlikely that more people with this same job description bump into each other in the same company. Controlling, reporting, data engineers, these are all “multiple repeating’s” jobs in the same firm. But sophisticated analytics is often rather lonely. As a result, deserted Data Scientists do not experience team spirit, they have nobody to consult with their assumptions of uncertainty.

Second factor heating the loneliness is absence of managerial leadership. Sophisticated analysts often find themselves within teams that deal with data only marginally (e.g. sales or marketing). Or the teams work with data, but just to report on structured databases. As a result the Data Scientists do not receive proper feedback on their work and are set to learn only from own mistakes (which they have to detect themselves the first place). Some torture themselves with self-study portals and courses, but few have the iron self-discipline to do these for several years in row. Sooner or later, the majority just throws the towel.

The two above mentioned forces are joined by the lack of external know-how as the third factor. No offence here, but if you want to experience conference where each speaker of the day contributed to your growth, you probably have to organize one by yourself. No jokes here, trust me, my own experience speaks here. Expert conferences are rarely organized by people who can tell if the speaker is beneficial or not. Most of the Data Science events are vendor-brainwashing or people-headhunting traps. That is why (in the popularity ever rising) Meetup’s are often the only remedy for the expert conference hunger.

How to brake the vicious circle?

Even though our dispute might not suggest so, there is a happy-end for the 4 lonely candidate stories. Although they represent a sad probe into soul of the contemporary data analyst, they also show a way how to brake the vicious circle. Based on their talks and my own expert experience I would suggest you following 3 steps out of the loneliness:

Step 1: Self-diagnostics. The real change can happen only if participants admit there is an issue to address. This makes the change needed and unavoidable. Thus, please give yourself following 3 questions: 1] Do I have a boss, who understands my job to extent that (s)he would be able to temporarily step in during my absence? 2] Is there somebody else in our company that I can ask if I am doing my Data Science tasks correctly or give me tips if I got stuck? 3] Did I have a chance to get my hands dirty on at least 2 new analytical approaches that I never tried before over the last 6-9 months? If at least one “NO” emerged for you in previous three questions, you should seriously think about your analytical future.

Step 2: Deep breath before the dive. If you self-diagnosed yourself for a move ahead, do not run to job portals to look for new job ads. Before you embrace the switch, get ready for the leap. If you start looking for a new job now right away, you will most likely fail to get one. Grab a bit of self-discipline. Take some on-line courses, watch YouTube videos on your new desired expertise. But foremost, force yourself to really try by your own hands the new skills that you will need in new job. Maybe start here.

Step 3: Look for real Data Science leader in real Data Science team. To escape the analytical loneliness for real, one has to solve all the three underlying factors. Therefore, .. (a) you have to find a job, where you will be .. (b) part of the larger team working on Data Science and the team … (c) will be led by person with own, real analytical experience. Team that has ambition to work on plenty of new, analytics relying projects. Don’t get fooled by sexy offers of companies where one of the 3 aspects is hyped. Start-up’s often look as cool place to work, but often is accompanied with inexperienced founder who “just” had a great idea OR dire DYI conditions that leave you weak to unreliable systems or unskilled neighbor teams. I advise you to start the search by nailing down interesting projects and check on who is leading them. Alternatively, you can start your search from respected analytics manager and check is his team works on something that would make you dream big again. With high quality managers do not bother to revisit your former bosses’ profiles as well. As they say in airplane security instructions: “Look around, as your nearest emergency exit may be located behind you as well”.

On final note, let me emphasize that analytical loneliness might be a cyclic phenomenon. As the Data Science industry takes traction, teams will grow and finally form generation of nature Data Science leaders. However, in our region it make take well 5-7 years before happening so. Hence, probably a bit too long of a period to “shelter against the storm” in your recent hiding. Thus, if you identified yourself with some aspects of analysts’ loneliness, do not sooth yourself, it will get better sometime. Repeatedly do the same stuff, in stable, well-payed job, where my boss does not know enough to mess into my job or to fire me, can sound like recipe for nice life. So rather than galvanizing you to action, I ask you to give it a thought: How will the whole industry shift in the meantime? How do my chances to switch to more sophisticated analytics improve/worsen as I enjoy the “invincible” times? No matter if you decide to stay or get ready for the leap, let me wish you months (or years) without analytical loneliness.

Späť na domovskú stránku