Bibliographic record
Abstract
Today, one of the most important concerns of information systems, is the processing of massive database queries.Parallel processing system of Google called Map Reduce and open source version of it called hadoop have the ability to work on multiple concurrent execution processing systems.Despite the relatively high performance still it needs to improve the processing method when dealing with the processing of the giant databases.SQL is a famous and powerful database that can store large amounts of data, maintain and make access to the information required for the processing.The database, contain an order called Hash Join that is one of the ways of implementation of join between the tables of a database.In this study, a method is suggested for executing Hash Join in Map Reduce that increases the speed of the join between the tables and the outputs.By comparing this technique with the usual implementation of Hash Join on Map Reduce system, it can be seen that approximately 30 percent improvement is achieved through the proposed approach in running the system.Keywords: distributed system of Map reduce -hadoop Software -query processing -SQL Database, hash join 1. Introduction Obviously, one of the most time-consuming phases in a database system is query evaluation engine particularly when there are high volumes of data.Rapid and effective assessment of information queries in a space which is faced with an explosion of information, increases system performance.The use of higher processing power and exploitation of the parallel processing is one of the primary mechanisms.Although many efforts have been done to parallelize query evaluation, the investigation is still open in this regard.But when the volume of data increases, For example, when dealing with the data on the volume of Tera or Peta, the issue is made more complex.But with such massive amounts of data, we are needed to evaluate the query algorithms from different perspectives.Query algorithms, are extremely important in the database management and can help connect the tables by adopting special conditions.2. A review of literature An article titled Google's programming model in Map reduce was conducted in [1].Then it was revised in article [2] which examined the model and provided suggestions for improving it.However, despite the usefulness of this article (in terms of programming model for processing actions such as join analysis and structural use of it) it has not mentioned anything about improving processes in the structure.An article is offered entitled as an optimized framework for queries [3] of Map Reduce.In this paper, a framework is suggested for effectively optimizing semi SQL queries of Map Reduce.It is a new query based on algebra, and it uses a small number of high level physical operators that are independent from existing systems of Map Reduce, such as hadoop.Algebra used in this article, and therefore its processes are relatively simple and effective and this is its strength.But due to the small number of these operators query times are higher and a greater number of machines are distributed and involved in implementing the join.An article entitled Hive: a
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.002 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.003 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.004 | 0.001 |
| Insufficient payload (model declined to judge) | 0.935 | 0.955 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".