Sorry Canada, your IT failed.

[mynameisjuan]

2 minutes ago, samcool55 said:

Also, how tf did you end up in a position in a large company where you could demand hard dates? Our IT department isn't 1 person but nowhere near enough to offer actually proper support.

Our IT dept had not only a lot of say but we also had a lot of power. The directors listened to us and what was needed to be done, it was the best place to work honestly, I will never find that type of management again. 

 

The company also followed very strict laws and was audited bi-weekly so I made sure everything was kept locked down and secure or it would be hit with a $50,000 fine in some cases. 

7 minutes ago, samcool55 said:

Funny detail: some issues went all the way up to the chain, to the big boss, and he actually gave a green light for multiple things that would help us, but the rest of management said no, so nothing happened...

As I just wrote, I will never find a job when IT gets the say. It was glorious and I had that place with next to 99.9% uptime. Management for the most part gives no shits about IT. 

[GoodBytes]

2 minutes ago, mynameisjuan said:

 As I just wrote, I will never find a job when IT gets the say. It was glorious and I had that place with next to 99.9% uptime. Management for the most part gives no shits about IT. 

When things works, Management says "Why we need an IT department?"

When things don't work, Management gets angry and wonders: "Why do have an IT department, nothing works!"

[mynameisjuan]

Just now, GoodBytes said:

When things works, Management says "Why we need an IT department?"

When things don't work, Management gets angry and wonders: "Why do have an IT department, nothing works!"

Cant be more true....:(

[samcool55]

33 minutes ago, mynameisjuan said:

Our IT dept had not only a lot of say but we also had a lot of power. The directors listened to us and what was needed to be done, it was the best place to work honestly, I will never find that type of management again. 

 

The company also followed very strict laws and was audited bi-weekly so I made sure everything was kept locked down and secure or it would be hit with a $50,000 fine in some cases. 

As I just wrote, I will never find a job when IT gets the say. It was glorious and I had that place with next to 99.9% uptime. Management for the most part gives no shits about IT. 

An audit is iirc planned, but not soon. Maybe it will help giving us the leverage getting the green light from management...

 

Be happy that you had such an awesome place because there are a lot of places out there that are worse.

[mynameisjuan]

8 minutes ago, samcool55 said:

An audit is iirc planned, but not soon. Maybe it will help giving us the leverage getting the green light from management...

 

Be happy that you had such an awesome place because there are a lot of places out there that are worse.

Use it to leverage when you can. Ask the auditor (if you speak to them directly) for fines in place or loss of licensing if something fails. 

 

But it was the best place to work beside pay. I do miss it

[Misanthrope]

44 minutes ago, GoodBytes said:

When things works, Management says "Why we need an IT department?"

When things don't work, Management gets angry and wonders: "Why do have an IT department, nothing works!"

That's why it's important to be "that guy" that makes you roll your eyes just thinking about it.

 

I usually am that guy, even at my work. But it serves a purpose and since professionally I know exactly how to present this challenges I usually don't get in trouble but I do confront superiors, even higher up management and tell them "Look this is what you're asking for, this is why it can't be done with what we have in this much time, it's going to take this much time instead and you need to either make some fundamental changes to your data or adjust your expectations because you just don't have enough employees or programmers to just make things happen in the deadline you're throwing around"

[Yoinkerman]

Its almost like it was pushed through so certain people could make scads and scads of money 

[PocketNerd]

This isn't really new news if you live in Canada.

Pheonix was rolled out at least a year ago and is still failing. Like, why the fuck did they not do a phase-in for the new system so they can catch the issues?!

[GoodBytes]

7 minutes ago, PocketNerd said:

This isn't really new news if you live in Canada.

Pheonix was rolled out at least a year ago and is still failing. Like, why the fuck did they not do a phase-in for the new system so they can catch the issues?!

It is because MANY people don't know what software development is. They think that ordering a software, is like ordering a stapler. A simple, physical good, that was perfected and matured for a number of years, and once you release it, it means that it is 100% complete and all facets were properly implemented, and you also figured out missing requirements that are departmental specific that was forgotten. Example, you are asked to make a stapler for stapling 2 to 10 sheets, you did, but also designed it to take 0 to 100 sheets and hope that some department doesn't end up stapling 101 sheets and your stapler jams, all without costing to you a penny more, somehow.

[PocketNerd]

48 minutes ago, GoodBytes said:

It is because MANY people don't know what software development is. They think that ordering a software, is like ordering a stapler. A simple, physical good, that was perfected and matured for a number of years, and once you release it, it means that it is 100% complete and all facets were properly implemented, and you also figured out missing requirements that are departmental specific that was forgotten. Example, you are asked to make a stapler for stapling 2 to 10 sheets, you did, but also designed it to take 0 to 100 sheets and hope that some department doesn't end up stapling 101 sheets and your stapler jams, all without costing to you a penny more, somehow.

That argument makes sense for the average joe. But this is the Federal Government, with Degree and Certification requirements for employment.

Having actually worked for a branch of the Federal Government for a year (Canadian Grain Commission, started out as a student job) in IT, there are people there who actually know what they're doing. So how could a fuck-up this big occur?!

Like seriously, we did a phased change-over for swapping all of the Printers in the Grain Elevators in my region. This shouldn't be rocket science!

[GoodBytes]

14 minutes ago, PocketNerd said:

That argument makes sense for the average joe. But this is the Federal Government, with Degree and Certification requirements for employment.

Having actually worked for a branch of the Federal Government for a year (Canadian Grain Commission, started out as a student job) in IT, there are people there who actually know what they're doing. So how could a fuck-up this big occur?!

Like seriously, we did a phased change-over for swapping all of the Printers in the Grain Elevators in my region. This shouldn't be rocket science!

These decisions doesn't come from people your manager or even upper manager of the IT department you were working in.

It is all the way on the top, which may or may not know. And it is not the first time that an upper manager doesn't listens to its advisers/managers. We see this ALL the time. How many engineer reports comes out saying that some bridge will collapse tomorrow, and nothing is done for 15 years?  Because people on top goes "You are exaggerating, it is fine, it looks fine... " and then once parts start falling on people and cars bellow the next day, they go "I was miss informed, terrible engineers! Daring to make me look bad! We will start some emergency fixing plan, and we will look into building a new bridge". Good thing that no one died... oops: https://en.wikipedia.org/wiki/De_la_Concorde_overpass_collapse

 

[PocketNerd]

7 minutes ago, GoodBytes said:

These decisions doesn't come from people your manager or even upper manager of the IT department you were working in.

It is all the way on the top, which may or may not know. And it is not the first time that an upper manager doesn't listens to its advisers/managers. We see this ALL the time. How many engineer reports comes out saying that some bridge will collapse tomorrow, and nothing is done for 15 years?  Because people on top goes "You are exaggerating, it is fine, it looks fine... " and then once parts start falling on people and cars bellow the next day, they go "I was miss informed, terrible engineers! Daring to make me look bad! We will start some emergency fixing plan, and we will look into building a new bridge". Good thing that no one died... oops: https://en.wikipedia.org/wiki/De_la_Concorde_overpass_collapse

 

And that's when you know that those people need to be sacked.

[Tribalinius]

3 hours ago, leadeater said:

@Tribalinius @wkdpaul

Another issue that really gets to me is the way tender processes are done to select the software vendor to go with. There are a lot of things you aren't allowed to do or steps that don't get done that seem rather fundamental to enabling the selection of the best fit product.

 

Like for our new Student Management System, it was all done in secret and none of the infrastructure support team or application support team were allowed to know what the options were so we could investigate how the proposed systems would fit in to our environment, what systems it would need to interact with and how. How are you supposed to make an informed decision and pick the best product that is actually supportable when you never bother to find those things out, yes the business side of it is the most important by far but it does you no good to select a product that just will not work no matter how much shoehorning you try and do.

 

Then to top that off no one is allowed to ask any other businesses that use the proposed software what it's like, how they implemented it, what issues they encountered, nothing.

 

The above project was multiple tens of millions of dollars, not allowed to say exactly how much and was literally dropped on us over night. Not kidding the first we found out a product was selected was when they walked in and demanded 100 VMs be created immediately and the project was starting that day. 

 

Good decisions can not be made in a vacuum.

@wkdpaul @leadeater

Secret stuff is the worst, I've been in companies structure where top brass expects a circle to connect through a square to a triangle without talking to anyone months after deals are signed. That's one of the reason I quit regular IT infrastructure jobs. Geez, we're your IT department, we're the most qualified to tell your decision and timeline to implant it sucks donkey balls! 3 words: LISTEN. TO. US. It's so infuriating when you have to deal with people taking decisions in your back than they start to whine because nothing is working the day after. It's our job to evaluate products in your work environment and the impact of your decisions on your business. That's a total lack of respect to not include us in these meetings, force us on something not planned and sling shit at us right after because it's not working at your liking.

 

I don't have a lot of perks being an external consultant but freedom of speech is one of them. I don't have to hold back in these meetings anymore and I'm not afraid anymore to confront business owners and tell them what I really think about a particular project. It gives me the flexibility and the leverage I did not have before. I've saved a couple of businesses from making costly terribad decisions because they listened the advice/opinion we gave them. You don't want to listen? Your call buddy. I'll be waiting for your call to plug the hole with my magical IT glue when the ship starts to sink. I've been part of that show long enough to know the outcome and the cost of badly managed projects lol.

You are right when you say that good decisions can not be made in a vacuum but that vacuum should consists of the key people implicated in these projects. Too many people or the wrong people sitting around the table can steer a project in the wrong direction quite fast.

 

[Tribalinius]

15 minutes ago, GoodBytes said:

These decisions doesn't come from people your manager or even upper manager of the IT department you were working in.

It is all the way on the top, which may or may not know. And it is not the first time that an upper manager doesn't listens to its advisers/managers. We see this ALL the time. How many engineer reports comes out saying that some bridge will collapse tomorrow, and nothing is done for 15 years?  Because people on top goes "You are exaggerating, it is fine, it looks fine... " and then once parts start falling on people and cars bellow the next day, they go "I was miss informed, terrible engineers! Daring to make me look bad! We will start some emergency fixing plan, and we will look into building a new bridge". Good thing that no one died... oops: https://en.wikipedia.org/wiki/De_la_Concorde_overpass_collapse

 

It always come down to $$$... "How far can we stretch it out before a catastrophe happens" is one way to deal project management I guess

[leadeater]

38 minutes ago, Tribalinius said:

Too many people or the wrong people sitting around the table can steer a project in the wrong direction quite fast.

"But all we need is more people to bail water out of the sinking boat"

"Keep bailing, faster, more, we'll make it"

*ship sinks*

 

[mr moose]

8 hours ago, samcool55 said:

The issue is that due to the way Windows 10 handles upgrades. Every 6 months you risk stuff breaking again.

Today a few systems that run W10 upgraded to 1803. All of them broke... It was an easy fix but still, it was downtime that wouldn't have happend if it updated the same way like W7 and W8.

That is the exact reason updates and upgrades can be disabled or handled i house on enterprise (education too I believe) versions of windows.   fleet managers like to see the effects of updates in the wild or on a few controlled systems before mass release to their fleet.  

[mr moose]

1 hour ago, leadeater said:

"But all we need is more people to bail water out of the sinking boat"

"Keep bailing, faster, more, we'll make it"

*ship sinks*

[mark_cameron]

11 hours ago, TechyBen said:

Canada's latest attempt at a unified Payroll system has imploded. It's sad to see IT projects fall apart. Either through budget problems, feature creep or lack of knowledge and understanding.

 

http://www.theregister.co.uk/2018/05/29/canada_phoenix_payroll_system_audit/

 

 

So a massive extra cost just to rip it out and start again. I know it's a bit of old news, as this is just a confirmation that it failed by the auditors at this point. But it got me wondering about things like how tiny little companies can get it right or wrong, in comparison to national or global companies getting it right or wrong. I guess scale makes a massive difference.

 

So the advice to LTTs and their "IT" systems? Keep it simple, because the bigger it is, the more you can drop!

 

[edit]

I just realised... this one needs to rise from the ashes! 

"Not enough Canadian Rubles"

 

[WkdPaul]

1 hour ago, leadeater said:

"But all we need is more people to bail water out of the sinking boat"

"Keep bailing, faster, more, we'll make it"

*ship sinks*

 

Yeah, that's pretty much what happened at my previous employer, after they changed the inventory and billing software. The biggest department impacted was IT and logistics, the guys in the warehouse went from taking 5 to 10 minutes per order (from picking to packing) to 15-20 minutes ... that's without counting all the BS around the inventory management bugs and shortcomings this new software had!

 

Management never once acknowledged any problems (even in the face of tons of criticism from clients and even after losing big clients they kept shouting on the rooftops that everything was fine and we were doing great), but the policy changes that followed shows that they knew they fucked up : they had a hiring freeze policy, but after that new software came live, all branches logistic department were given the green light to get "temps" from the local employment agencies.

 

Cost went up, sales tanked ... 2 years later, they went from 4 branches in Canada to only 1! Congrats 

Edited May 30, 2018 by wkdpaul

[Lurick]

4 minutes ago, wkdpaul said:

-snip-

Just recently dealt with a sales guy who literally said "Why do we need a dedicated lab for this customer? Why do we need test tools? Let's just give it all to the general lab so we don't pay any more depreciation and we'll spin up labs when we need it at random. That won't cause any problems and you can get it done at the snap of a finger when the customer asks, right?" We about knocked that moron so hard up the head -.-*

[WkdPaul]

17 minutes ago, Lurick said:

Just recently dealt with a sales guy who literally said "Why do we need a dedicated lab for this customer? Why do we need test tools? Let's just give it all to the general lab so we don't pay any more depreciation and we'll spin up labs when we need it at random. That won't cause any problems and you can get it done at the snap of a finger when the customer asks, right?" We about knocked that moron so hard up the head -.-*

Yes, we all know a moron like this in some form or another, and one (or rather multiple) of these is probably the reason this thread exists and why the Phoenix payment system crashed and is burning a hole in Canadian's tax payers pockets... 

 

 

EDIT : @leadeater @Lurick @Tribalinius oh yeah, and totally forgot to mention that, but any of your guys heard of the Peter principle? ;

https://en.wikipedia.org/wiki/Peter_principle

 

Seems to apply to the discussion at hand

Edited May 30, 2018 by wkdpaul

[Bcat00]

7 hours ago, PocketNerd said:

And that's when you know that those people need to be sacked.

Sad to say, it’s almost impossible to sack those working in government positions unless you deliberately went out of your way to get fired.

[Canada EH]

18 hours ago, TechyBen said:

So a massive extra cost

Yup

 

Gotta thank the government for waste!

[TechyBen]

12 hours ago, mark_cameron said:

"Not enough Canadian Rubles"

 

I'd trust keyboard cat over most. I have tuna. Tuna is cats friends. I can haz softwearze now kthxbye?

[Tribalinius]

14 hours ago, wkdpaul said:

Yes, we all know a moron like this in some form or another, and one (or rather multiple) of these is probably the reason this thread exists and why the Phoenix payment system crashed and is burning a hole in Canadian's tax payers pockets... 

 

 

EDIT : @leadeater @Lurick @Tribalinius oh yeah, and totally forgot to mention that, but any of your guys heard of the Peter principle? ;

https://en.wikipedia.org/wiki/Peter_principle

 

Seems to apply to the discussion at hand

Never heard of that before but it makes total sense!