Im looking for some advise on a work project

titaniumtechconsulting · September 9, 2020

Hey guys, there aren't a lot of communities that i can trust to ask this of, but i am trying to collect the data from a ecommerce site for my work. we work in contracting and need a database of the prices and the item that can be searched and updated relatively easily. now i could always open each individual item and copy paste the name and price but I am hoping that some can advise me of a solution that can be implemented that would allow me to scrape the data that I want and save it into something like a spreadsheet., currently with price quotes we waste so much time calling and emailing the company for product pricing that i would like to stream line the process and have a copy of these prices on my work computer. I hope someone can steer me in the right direction for this project. thank you obi wans (guys), you are my only hope!

LogicalDrm · September 10, 2020

-> Moved to Programs, Apps and Websites

***

So, whats your actual question? Like recommendations on some premade database solutions? Since you could do it in-house using some SQL database system, setup queries and stuff like that yourself.

titaniumtechconsulting · September 10, 2020

how would i go about getting the data like product title and price data for each item and have it put into a database or spreadsheet without having to manually copy paste all 10000 entries. each product has its own link so it a lot of labour hours to get something like this done. if there is a way it could be done any advise would be of great help to me.

LogicalDrm · September 11, 2020

16 hours ago, titaniumtechconsulting said:

how would i go about getting the data like product title and price data for each item and have it put into a database or spreadsheet without having to manually copy paste all 10000 entries. each product has its own link so it a lot of labour hours to get something like this done. if there is a way it could be done any advise would be of great help to me.

I have used databases to do this, but only with very customized situations. So I was working for mining company, moving their database from software to another. Used Excel and Access for that. Pretty much the way to do it was that prepared Excel, with column titles and IDs. There were 5-7 tabs/tables with different column titles. Saved as CSV, then premade tables in Access to match column titles, correct types for columns. Then just imported CSV file. In Access connected IDs between tables. The main software uses Access databases, and all searches are done from there even when Access should be used for adding or updating data as its more stable than the actual software.

So essentially you can use Excel for preparing data and then use CSV as transferring format to any database system. Updating prices and stuff like that should be done with form making updating query to correct record. If you would have some suite for it, they would have tools made for the job. Like what I have used with Surpac and ArcGIS.