August 24, 2016 at 9:29 pm
Comments posted to this topic are about the item Better Coding
August 25, 2016 at 12:35 am
I don't always read the referenced articles but today's proved to be ever so interesting. Considering the editorial, the referenced articles and the comments that followed I have come to the conclusion that:
b) the students' solution is impractical but highlighted options.
I was looking for comments along the lines of offloading the validation to the client and using offline server side processing like Amazon. Amazon was mentioned but the key to Amazon's scaleability wasn't: orders are processed offline.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
August 25, 2016 at 2:04 am
I have been on the receiving end of 3rd party partnerships with IT vendors.
I wanted to feel like a customer, not a conquest.
After the high-ups have been wined & dined, signed the contract for 'n' years the honeymoon period is woefully brief. The high-ups would lose too much face to admit they'd been hoodwinked so unless the Vendor is stupid enough to breach the letter of their contract the vendor knows they can put their Z team in place for the rest of creation.
Not all vendors are like this. Some are honest enough to say that in order for the deal to be worth their while they need resources available for a specified time period and to complete the work by a specific date. Personally my experience has been that the latter type tend to favour an Agile approach
August 25, 2016 at 2:33 am
For a census the app simply collects the post from a web form and persists it.
Given that it is a once every 10 year thing with fixed questions you are not far from having a write only app. Maybe you need something to allow people to go back to previous pages but beyond that I can't see the benefit in providing any application data retrieval.
So you have webforms (with validation), a back end validator that either rejects the post or persists it to a queue, a session store (REDIS is a candidate), the aforementioned queue, a queue reader to post the census into a DB.
SSL encryption end to end. An intrusion protection system. If anything you could pare the webservers down to the bare minimum required for this simple app. Disable all unused mods and plugins etc.
The webservers need to scale out, as does the queue.
Given that there is no immediate requirement for the data you could have different collections of webservers talking to different queues and DBs then merge the results post census day.
In AWS make sure your components are separated into separate subnets with security groups and Network ACLs applied.
Create an Amazon Machine Image for each hardware component so additional webservers can be spun up in seconds fully configured.
Put the hardware in autoscaling groups so if hardware fails it will be replaced automatically.
What have I missed?
August 25, 2016 at 5:34 am
David.Poole (8/25/2016)
For a census the app simply collects the post from a web form and persists it.Given that it is a once every 10 year thing with fixed questions you are not far from having a write only app. Maybe you need something to allow people to go back to previous pages but beyond that I can't see the benefit in providing any application data retrieval.
So you have webforms (with validation), a back end validator that either rejects the post or persists it to a queue, a session store (REDIS is a candidate), the aforementioned queue, a queue reader to post the census into a DB.
SSL encryption end to end. An intrusion protection system. If anything you could pare the webservers down to the bare minimum required for this simple app. Disable all unused mods and plugins etc.
The webservers need to scale out, as does the queue.
Given that there is no immediate requirement for the data you could have different collections of webservers talking to different queues and DBs then merge the results post census day.
In AWS make sure your components are separated into separate subnets with security groups and Network ACLs applied.
Create an Amazon Machine Image for each hardware component so additional webservers can be spun up in seconds fully configured.
Put the hardware in autoscaling groups so if hardware fails it will be replaced automatically.
What have I missed?
It appears simple enough. Why can't it work? Something for this big of an audience never seems to.
August 25, 2016 at 7:20 am
]It appears simple enough. Why can't it work? Something for this big of an audience never seems to.
In my experience the solutions are over engineered and under provisioned.
The over engineering tends to be vendors shoehorning in components for which they can charge handsomely but offer little relevant benefit and loads of complexity to the solution.
If you have 3 components it doesn't take long to find which one is a bottleneck. If you have 50 then it is massively harder to pin down a problem
August 25, 2016 at 7:36 am
The number of attack vectors for a public project is constantly changing. Any single one of them can be addressed, but rarely do we catch all of them.
412-977-3526 call/text
August 25, 2016 at 7:49 am
As a contractor who works on government contracts, I would suggest just blaming the contractor for the failings for government projects belies bias on your part and a misunderstanding of how governments award contracts.
Typically, they only award fixed bid contracts and with price being weighted to be 50% of the criteria. Almost always the cheapest bid wins.
Then they complain when they can't have a Cadillac on a Yugo budget. This then leads to a constant war against scope creep as they business tries to get what they need while the vendor is trying to cut costs so they can still make a profit on the project. This leads to acrimony and an inability to work efficiently on the project, further hampering the ability of the vendor to bring the project in on time.
August 25, 2016 at 7:52 am
David.Poole (8/25/2016)
...What have I missed?
$$$ (Aus, of course.)
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
August 25, 2016 at 7:54 am
Brett Phipps (8/25/2016)
...Then the complain when they can't have a Cadillac on a Yugo budget. This then leads to a constant war against scope creep as they business tries to get what they need while the vendor is trying to cut costs so they can still make a profit on the project. This leads to acrimony and an inability to work efficiently on the project, further hampering the ability of the vendor to bring the project in on time.
Plus the need to charge extra because governments around the world are often late payers so the finance needs refactoring.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
August 25, 2016 at 8:13 am
I'm not surprised that a small group of students could take a public dataset and spin up a better application than the original government IT contractors. When dealing with non-sensitive datasets, perhaps the government should approach this from the bottom up, rather than the top down. For example, start by publishing the data and a set of high level requirements, see who can create the best prototype solution that meets the requirements, and then award them a contract to finish off the project.
If some some bloke from Denver named Dan can create BI solutions better, faster, cheaper than a billion dollar corporation that specializes in manufactoring jet planes and electionics, then why not just hand the job over to Dan?
Here I'm just talking about the application development itself. The government could still farm out the infrastructure and support to the usual contractors.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
August 25, 2016 at 9:35 am
Eric M Russell (8/25/2016)
I'm not surprised that a small group of students could take a public dataset and spin up a better application than the original government IT contractors. When dealing with non-sensitive datasets, perhaps the government should approach this from the bottom up, rather than the top down. For example, start by publishing the data and a set of high level requirements, see who can create the best prototype solution that meets the requirements, and then award them a contract to finish off the project.If some some bloke from Denver named Dan can create BI solutions better, faster, cheaper than a billion dollar corporation that specializes in manufactoring jet planes and electionics, then why not just hand the job over to Dan?
Here I'm just talking about the application development itself. The government could still farm out the infrastructure and support to the usual contractors.
Because 'Dan' is designing it on his own...not following the Government's criteria. This is where the issues really happen. The government can be really specific on what they require. Read a government contract sometime. There's been lots in the news about the cost of the toilets on the shuttle or the space station. Also, costs of military aircraft/ships. But when the requirement includes a specific description of a wrench and that description means you have to create a process to make that wrench because the $5 one you can find in the store doesn't exactly match the requirement....there go the costs up and up.
-SQLBill
August 25, 2016 at 9:53 am
SQLBill (8/25/2016)
Eric M Russell (8/25/2016)
I'm not surprised that a small group of students could take a public dataset and spin up a better application than the original government IT contractors. When dealing with non-sensitive datasets, perhaps the government should approach this from the bottom up, rather than the top down. For example, start by publishing the data and a set of high level requirements, see who can create the best prototype solution that meets the requirements, and then award them a contract to finish off the project.If some some bloke from Denver named Dan can create BI solutions better, faster, cheaper than a billion dollar corporation that specializes in manufactoring jet planes and electionics, then why not just hand the job over to Dan?
Here I'm just talking about the application development itself. The government could still farm out the infrastructure and support to the usual contractors.
Because 'Dan' is designing it on his own...not following the Government's criteria. This is where the issues really happen. The government can be really specific on what they require. Read a government contract sometime. There's been lots in the news about the cost of the toilets on the shuttle or the space station. Also, costs of military aircraft/ships. But when the requirement includes a specific description of a wrench and that description means you have to create a process to make that wrench because the $5 one you can find in the store doesn't exactly match the requirement....there go the costs up and up.
-SQLBill
Agreed- as a friend who was a veteran pointed out to me: you go design a toilet that can be airdropped from 500 feet out of the back of a plane without a parachute, that won't break, doesn't require extensive assembly and can run without a water or power supply, and see what price tag YOU come up with :w00t:
----------------------------------------------------------------------------------
Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?
August 25, 2016 at 10:01 am
Matt Miller (#4) (8/25/2016)
SQLBill (8/25/2016)
Eric M Russell (8/25/2016)
I'm not surprised that a small group of students could take a public dataset and spin up a better application than the original government IT contractors. When dealing with non-sensitive datasets, perhaps the government should approach this from the bottom up, rather than the top down. For example, start by publishing the data and a set of high level requirements, see who can create the best prototype solution that meets the requirements, and then award them a contract to finish off the project.If some some bloke from Denver named Dan can create BI solutions better, faster, cheaper than a billion dollar corporation that specializes in manufactoring jet planes and electionics, then why not just hand the job over to Dan?
Here I'm just talking about the application development itself. The government could still farm out the infrastructure and support to the usual contractors.
Because 'Dan' is designing it on his own...not following the Government's criteria. This is where the issues really happen. The government can be really specific on what they require. Read a government contract sometime. There's been lots in the news about the cost of the toilets on the shuttle or the space station. Also, costs of military aircraft/ships. But when the requirement includes a specific description of a wrench and that description means you have to create a process to make that wrench because the $5 one you can find in the store doesn't exactly match the requirement....there go the costs up and up.
-SQLBill
Agreed- as a friend who was a veteran pointed out to me: you go design a toilet that can be airdropped from 500 feet out of the back of a plane without a parachute, that won't break, doesn't require extensive assembly and can run without a water or power supply, and see what price tag YOU come up with :w00t:
Like I said, the government would publish requirements (both internal and public facing) for the web application along with the dataset. If the data is aggregated properly, then the securing the application is straightforward. The analogy of an indestructable military grade hammer doesn't pertain to a public website for visualizing census data. If someone thinks it does, then that's part of the problem.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
August 25, 2016 at 10:07 am
Federal contractors, at least within the realm of IT, farm the actual work out to sub-contractors anyhow, and I can tell you that those sub-contractors are no more skilled than the rank and file IT folks you would find in corporate America or fresh university graduates.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
Viewing 15 posts - 1 through 15 (of 20 total)
You must be logged in to reply to this topic. Login to reply