Computer hardware analytic for electronic commerce

VIETNAM NATIONAL UNIVERSITY - HO CHI MINH CITY HO CHI MINH UNIVERSITY OF TECHNOLOGY COMPUTER SCIENCE AND ENGINEERING FACULTY GRADUATION THESIS Computer Hardware Analytic for Electronic Commerce Department: Computer science Committee: Advisor: Reviewer: Students: Computer Science Dr Phan Trong Nhan Assoc Prof Nguyen Thanh Binh -o0o Ly Gia Bao 1752089 Tran Nguyen Anh Huy 1752244 Tran Anh Thai 1752494 Le Huynh Long Vy 1752637 HO CHI MINH CITY, 12/2021 - - KHOA:KH & KT Máy tính HTTT ình Lý Gia B MSSV: 1752089 Tr MSSV: 1752244 Tr _ MSSV: 1752494 Lê Hu g V _ MSSV: 1752637 NGÀNH: Khoa h áy tính _ Computer Hardware Analytics for Electronic Commerce Surveying and analyzing related systems Finding out hardware data sources for analytics Proposing core features for the whole system towards recommendation, association rules, personalization, segmentation, and trends of hardware combination Identifying main academic problems to be solved Researching relevant theory (mainly association rules, prediction, recommendation techniques) and practical approaches for the core features implementation as well as for those issues Researching in-need technologies Designing the system with the proposed features and solutions Developing a system prototype Implementing and perfect the system Conducting empirical experiments 23/08/2021 13/12/2021 1) TS Phan Tr ân Ngày 23 tháng 08 2021 PGS TS Tr _ _ _ TS Phan Tr ân KHOA KH & KT MÁY TÍNH -Ngày 20 tháng 12 2021 ) Lý Gia B MSSV: 1752089 Ngành (chuyên ngành): Khoa h Tr MSSV: 1752244 Ngành (chuyên ngành): Khoa h Tr MSSV: 1752494 Ngành (chuyên ngành): Khoa h Lê Hu gV MSSV: 1752637 Ngành (chuyên ngành): Khoa h Computer Hardware Analytics for Electronic Commerce : TS Phan Tr ân áy tính áy tính áy tính áy tính - Students developed a system that analyzes computer components score bands and recommends the combination of them, especially with their compatibility while there are many choices, to meet the end-users need Both auto and manual modes with advanced options are supported Moreover, the users can customize their options as they wish Even when the result is not found, the recommendation module will be automatically triggered - Students analyzed problem statement as well as its related work and challenges - Students applied modern technologies such as Django/python compatible with MVC pattern, redis, selenium, scrapy to implement the system and deployed the database on Google Cloud Platform - Students were very proactive and open to novel knowledge and technology LVTN: - The evaluation of recommendation and system testing is still limited - Some features need to improve such as dashboard with more metrics and component choices with further personalization, flexibility, and reliable benchmark scores Very good 9.5/10 Phan Tr ân Ho Chi Minh City University of Technology Faculty of Computer Science and Engineering SOCIALIST REPUBLIC OF VIETNAM Independence Freedom Happiness REVIEW OF GRADUATION THESIS HCMC, 28th December 2021 Students name: 1752089) 1752244) Major: Computer Science Thesis title: Computer Hardware Analytic for Electronic Commerce Reviewer: Assoc.Prof Nguyen Thanh Binh, PhD Contents - In this thesis, the authors built a system helping the users to build a computer even if they have no experience, while simultaneously providing an alternative for those who want to pick and choose the components for their own computer The tasks in this thesis: + Build a web-server to interact with users + Build a collection of data from multiple sources + Build a data storage + Develop an algorithm that will automatically generate a PC based on the user's requirements + Dev - The authors should explain more system which the authors built The system has disadvantages for users - Some spelling errors in the essay - The author should clarify the advantages of the system which was built compared to existing systems General assessment: Marks 8.5/10 Reviewer Nguyen Thanh Binh Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Declaration We guarantee that this research is our own, conducted under the supervision and guidance of our Instructor Phan Trong Nhan The result of our research is legitimate and has not been published in any forms prior to this All materials used within this research are collected by ourselves, by various sources which are appropriately listed in the references section In any case of mistake, we will take full responsibility for it Graduation Thesis Page /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Acknowledgements We would like to express our deepest gratitude to our advisor Dr Phan Trong Nhan for the continuous support and guidance during the making of our thesis Besides Dr Nhan, we would also like to thank all of our faculty lecturers, who have given us valuable and necessary knowledge to accomplish this project Finally, we wish you health, wealth, and success in your endeavors Graduation Thesis Page /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Abstract Nowadays, in the age of technologies, a personal computer (PC) is becoming the most crucial part of our everyday life However, for people who not have a solid understanding or familiarity with the area, purchasing or building a proper computer for a person’s needs can be difficult Therefore, it is an urgent need to research and develop a system that can take care of collecting the components’ data and help the user to build a computer that meets their requirements In this topic, we are trying to create a system that focuses on helping the users to build a computer even if they have no experience, while simultaneously providing an alternative for those who want to pick and choose the components for their own computer The system’s major goals will be to create a computer based on the user’s requirements, collect necessary data, and store it Furthermore, using the information gathered from user interactions, we will create a simple recommendation system that will identify some of the most popular PCs and have them suggested to users Graduation Thesis Page /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Content Introduction 10 1.1 Problem statement 10 1.2 Goal 11 1.3 Scope 11 1.4 Thesis structure 11 Methodologies and Theoretical Background 12 2.1 Related work 12 2.1.1 Manual PC builders 12 2.1.2 Automatic PC generator 12 2.1.3 PC performance calculator 12 Crawler 13 2.2.1 Selenium 13 2.2.2 Scrapy 13 Web server 15 2.3.1 Django 15 2.3.2 Redis 17 Google Cloud platform 17 2.2 2.3 2.4 System Requirement Analysis 18 3.1 Problems 18 3.1.1 Data gathering 18 3.1.2 Building PC 19 3.1.3 Recommendation system 19 Functional requirement 19 3.2.1 Automatic computer builder 20 3.2.2 Manual computer buider 20 3.3 Non-Functional requirement 20 3.4 Diagram 21 3.4.1 Use-case 21 3.4.2 Use-case Specification 22 3.2 System Design 34 4.1 General Architecture 34 4.1.1 How the system work 34 4.1.2 System architecture 34 Crawler component 35 4.2 Graduation Thesis Page /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer 4.3 4.4 4.5 Database Component 37 4.3.1 Database schema 37 4.3.2 Detailed information of each entity in the ERD 37 Web server Component 48 4.4.1 General architecture 48 4.4.2 Build PC module 50 4.4.2.a Checking the compatibility between the components 50 4.4.2.b Build a PC from the user’s requirements 53 4.4.3 Database module 57 4.4.4 Redis 57 Recommendation system 58 System Implementation 59 5.1 Technologies used 59 5.2 Building a Prototype 60 5.2.1 Home Page 60 5.2.2 PC Auto Generator Page 60 5.2.3 PC Auto Generator Result Page 61 5.2.4 PC Manual Generator Page 65 5.2.5 Customize PC Set Page 66 5.2.6 View PC Set Details Page 68 5.2.7 Search Product Page 69 5.2.8 Data Visualization 69 System Testing 70 6.1 Non-functional Testing 70 6.2 Functional Testing 72 Conclusion 76 7.1 Achieved result 76 7.2 Evaluation 77 7.3 Future development 77 Graduation Thesis Page /83 List of Figures 1.1 Budget distribution for gaming PC [3] 10 2.1 Scrapy architecture [13] 14 2.2 Django Architecture [7] 15 pattern [7] 2.3 MTV 15 3.1 Use-Case diagram 21 3.2 Use-Case diagram 22 4.1 General architecture 34 4.2 General idea of index page and product page 35 4.3 The flowchart of crawling 36 4.4 Database schema 38 4.5 Web server general architecture 49 4.6 Checking compatibility between the CPU and motherboard 51 4.7 Intel CPU generation 52 4.8 Workflow for auto building PC 54 4.9 How PC scores are calculated [16] 56 4.10 Workflow with data from Redis 58 5.1 Home Page 60 5.2 PC Auto Generator Page 61 5.3 Expand Advanced options 61 5.4 Generate successfully 62 5.5 Expand PC 63 5.6 Generate fails 64 5.7 Recommendation when generate fails 64 5.8 PC Manual Generator Page 65 5.9 Modify RAMs 66 5.10 Customize PC 67 5.11 PC Details Page 68 5.12 Search Product Page 69 5.13 Dashboard 70 6.1 Non-functional testing for the auto builder 71 6.2 Non-functional testing for the auto builder result page 71 6.3 Non-functional testing for the manual builder 72 6.4 Functional testing for the auto builder 73 6.5 Functional testing for the auto builder (cont.) 74 6.6 Functional testing for the manual builder 75 6.7 Functional testing for the recommendation system 76 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer and if the user is interested in purchasing, some components include connections to vendor pages In additional, user can get a copy of this computer set by clicking the ’Export as PDF’ button at the bottom of the page 5.2.7 Search Product Page Users can access pages corresponding to components to search for their products There are filters on each page that can be used to reduce the number of items to look for Furthermore, the user can sort the product list by column heading Figure 5.12: Search Product Page 5.2.8 Data Visualization User can view the dashboard to get general information of selected components such as price, benchmark, usage frequency, etc There are multiple interactions with the dashboard that user can take advantage of to get the preferred view: choose type of chart, choose number of components on the chart, display selected features Graduation Thesis Page 69 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Figure 5.13: Dashboard System Testing The website runs on local PC with specification: • Processor: Intelđ CoreTM i5-3470 CPU 3.20 GHz 3.60 GHz ã RAM: 12GB • Storage: 500GB • Operating system: Windows 10 Pro N 64-bit • Screen resolution: 1920x1080 The following figures contain some main testings for the system These tests are performed on the PC stated above 6.1 Non-functional Testing In this section, we test the usability, security of the website, performance and load testing of the auto builder Some other non-functional testings are listed in the appendix Graduation Thesis Page 70 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Figure 6.1: Non-functional testing for the auto builder Figure 6.2: Non-functional testing for the auto builder result page Graduation Thesis Page 71 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Figure 6.3: Non-functional testing for the manual builder 6.2 Functional Testing In this section, we test the functionality of the auto builder, manual builder, and the recommendation Some other functional testings are listed in the appendix Graduation Thesis Page 72 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Figure 6.4: Functional testing for the auto builder Graduation Thesis Page 73 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Figure 6.5: Functional testing for the auto builder (cont.) Graduation Thesis Page 74 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Figure 6.6: Functional testing for the manual builder Graduation Thesis Page 75 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Figure 6.7: Functional testing for the recommendation system 7.1 Conclusion Achieved result Through out this thesis proposal until now, we have learned and archived many ob- jectives Initially, through the research and analysis process, we have acknowledged and applied the web data crawling approach effectively We also encountered and overcame certain challenges during crawling Similarly, we have had experience in applying the Django framework and developed a functional website We’ve done our investigation and are able to create and use the database on the Google Cloud Platform And as a result of long periods of hard work, we have developed a framework that can examine the compatibility of PC components and generate a list of functional PCs based on the user’s requirements In the end, Graduation Thesis Page 76 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer we were able to accomplish the majority of the objectives we set out to archive at the start of the thesis 7.2 Evaluation In general, our website is functional and provides the majority of the intended func- tions The algorithm for automatically constructing a PC is capable of performing as planned Moreover, the data gathered is sufficient for the system to function and is correctly stored on the remote database And importantly, the system will be able to assist a user who has no prior experience developing a computer On the other side, one of the issues that has still to be resolved is that the data that we have gathered is still not diverse and is not updated on a regular basis Additionally, the user account is too simplistic and does not perform all of the tasks that other web pages 7.3 Future development • Deploy the system to an online platform • Improve the crawling process so that we can crawl more effectively Develop a behavioral data collector to deceive web servers, make it possible to collect data through the websites policies, and prevent anti-crawling or unexpected change in the structure of some websites • Develop a mechanism that helps update the products’ price frequently One way to deal with this is to use the Amazon Product Advertising API to update the products’ price so that user can build a better PC build with the up-to-date price • Improve the user account system so that they can get more personalized options and can save and share their favourite PC builds • Develop a system that can predict the trend of PC components in the future and the most used combination using data crawled from e-commercial websites in order to give more insights to users • Improve the build algorithm so that user can build with more options Graduation Thesis Page 77 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Reference [1] Amazon What Is an In-Memory Database? https://aws.amazon.com/nosql/inmemory/ Accessed: 2021-12-02 [2] Benchmarks A guide to benchmarks and benchmark scores https://benchmarks ul.com/guide-to-scores Accessed: 2021-12-02 [3] build-gaming-computers.com HOW TO BUILD GAMING COMPUTERS https: //www.build-gaming-computers.com/ Accessed: 2021-12-02 [4] Django Django at a glance https://docs.djangoproject.com/en/3.2/intro/ overview/ Accessed: 2021-12-02 [5] forums.tomshardware.com 1151 and 1151v2 https://forums.tomshardware.co m/threads/1151-and-1151v2.3388044/ Accessed: 2021-12-02 [6] Google Cloud Cloud SQL https://cloud.google.com/sql/?utm_source=goog le&utm_medium=cpc&utm_campaign=japac-VN-all-en-dr-bkws-all-supertrial-e-dr-1009882&utm_content=text-ad-none-none-DEV_c-CRE_5050118 53417-ADGP_Hybrid%20%7C%20BKWS%20-%20EXA%20%7C%20Txt%20~%20Databas es%20~%20Cloud%20SQL_cloud%20storage-google%20cloud%20sql-KWID_437 00023274810678-aud-970366092687%3Akwd-314195915211&userloc_1028581 -network_g&utm_term=KW_google%20cloud%20sql&gclid=CjwKCAiA78aNBhAlE iwA7B76p1FjhC9NtxoTNDGK99KCrR4UzucJf3vCohtQ1uy2KmUAknzm_9b_uhoChD0 QAvD_BwE&gclsrc=aw.ds Accessed: 2021-12-02 [7] Guru99 Django Tutorial for Beginners: Features, Architecture History https: //www.guru99.com/django-tutorial.html Accessed: 2021-12-02 [8] Guru99 Flask vs Django: What’s the Difference Between Flask Django? https: //www.guru99.com/flask-vs-django.html Accessed: 2021-12-02 [9] Intel Compatibility of Desktop 9th and 8th Generation Intel® Core™ Processors https://www.intel.com/content/www/us/en/support/articles/000025694 /processors/intel-core-processors.html Accessed: 2021-12-02 [10] Intel Intel® Desktop Processors Socket LGA1151 and LGA1200 Comparison https: //www.intel.com/content/www/us/en/support/articles/000056725/proce ssors.html Accessed: 2021-12-02 [11] Memurai About Us https://www.memurai.com/about Accessed: 2021-12-02 Graduation Thesis Page 78 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer [12] redis Introduction to Redis https://redis.io/topics/introduction Accessed: 2021-12-02 [13] Scrapy Architecture overview https://docs.scrapy.org/en/latest/topics/a rchitecture.html/ Accessed: 2021-12-02 [14] Scrapy Scrapy https://scrapy.org/ Accessed: 2021-12-02 [15] Selenium Selenium automates browsers That’s it! What you with that power is entirely up to you https://www.selenium.dev/ Accessed: 2021-12-02 [16] UserBenchmark What are the UBM performance classifications? https://www.user benchmark.com/Faq/What-are-the-UBM-performance-classifications/93 Accessed: 2021-12-02 Graduation Thesis Page 79 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Appendix: Additional testcases Non-functional Figure 7.1: Non-functional testing for navigation and login page Graduation Thesis Page 80 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Figure 7.2: Non-functional testing for register page Figure 7.3: Non-functional testing for detail page and product page Graduation Thesis Page 81 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Functional Figure 7.4: Functional testing for login page and register page Figure 7.5: Functional testing for the auto builder result page Graduation Thesis Page 82 /83 Ho Chi Minh City, University of Technology Department of Computer Science and Engineer Figure 7.6: Functional testing for detail page and product page Graduation Thesis Page 83 /83 ... Computer Hardware Analytics for Electronic Commerce Surveying and analyzing related systems Finding out hardware data sources for analytics Proposing core features for the... (chuyên ngành): Khoa h Computer Hardware Analytics for Electronic Commerce : TS Phan Tr ân áy tính áy tính áy tính áy tính - Students developed a system that analyzes computer components score... title: Computer Hardware Analytic for Electronic Commerce Reviewer: Assoc.Prof Nguyen Thanh Binh, PhD Contents - In this thesis, the authors built a system helping the users to build a computer