What is the most efficient way of problem-solving in programming
What is engineering?
“minimum cost maximum performance” sounds great but in programming, dimensions differ. Stability, elasticity, compatibility, performance yes this is a 4 dimension world.
At the end-user part, stable programs are more valuable than anything else. Maybe the reason consuming time, and time = money equation. no matter.
I will try to figure out similarities between real-life (3D) engineering and programming solutions.
Rule 1: Checkpoints.
Video game companies always use this strategy to keep the attention of their customers in the game. if something is broken turn back to the closest checkpoint start again with minimum loss.
Real-life engineering solution: Drinking water pumping over high points
Freshman solution
Senior Solution
Explanation: 10-meter water column generates a 1 bar pressure at the bottom and if you have to cross over a 300-meter hill you need to handle 30 bar pressure at water source level since no kind of water pipe can handle 30 bar ( max 12 bar ) engineers have to divide this pressure most equal and separate way. in the first figure (I still can not use photoshop and do not ask why, pls) pipe must generate high pressure, and this causes a weak point at water level and home level on the pipe.
This reminds me “one function, do all” type. anyway.
Programming Solution: web scraping
let's assume we need specific data on a website and we need to scrap 10.000 pages on this website.
Freshman solution
can you imagine the pressure on “for loop’’?
Senior Solution
Step 1: Using a database will be a good practice in any disaster situation gives the ability to start where you left, like a checkpoint huh?
Step 2: Raw data is important. You never know when you need it. Once you over-stress the network, it will be a good practice to save raw data. You can easily hash the link and use it as a file name so you can use it as a primary key between database rows and files you saved. Network connections have many unstable conditions, weak points and most of them are not on your side.
Step 3: Working with files instead of on-air network income data would be the most stable way of programming. who can take objection to this? It's time to scrap data with regex and save it to the database. Writing regex with super unconditional HTML codes needs some try-fail-again loops. Once you realize that your regex has some weak points and can not catch data. guess what, we already passed another checkpoint and we have no loss.
Step 4: Updating your database with each incoming data part will be another good practice. If something goes wrong you can easily start where you left last time with a very simple “ where table.data is NULL” query.
To summarize in a nutshell: checkpoints are olden but golden.
Happy coding :)
