And if we want to analyze data, we need to get it from somewhere, and we get it, as a rule, from databases. This is not always visible with the naked eye, but whether we read the news, transfer money from one account to another, request account statements, make purchases in an online store, or simply get acquainted with the prices of competitors – we do all this by referring to various databases.

Different levels and roles working with SQL or the results of its use

And before we get the usual “readable” representation of the information we need, the data is retrieved from the repositories. And most often this happens precisely with the use of SQL. In a broad sense, SQL is not a programming language, although there is such an established misconception. In fact, it looks more like the most common English speech.
Who are data analysts and what do they work with?

Data analysts are experimenters who own the tools to connect data streams from different sources, and also hypothesize and test them. This is where they need databases and a language that allows them to accurately formulate a query to them. Correct acquisition of the initial data is an art that guarantees a high probability that cause-and-effect relationships will be established in the behavior of the objects under study. It is the quality of the data that allows you to choose the most suitable hypotheses.

In addition to SQL, data analysts need to know tools for statistical data analysis: these are highly specialized packages – SPSS, Statistika, and various programming languages ​​- SAS, R, Python, which have functionality for data analysis and visualization, and very lightweight solutions like Gretl. And the most popular set of tools among analysts is, perhaps, Python + SQL.

Tools and Techniques Facing the Data Analyst

Often it is necessary to present the data in a visual and understandable way for the business, and sometimes it is necessary to evaluate for yourself which factors influencing the data are significant and which are not, which reasons for deviations are codependent. When it is necessary to make forecasts, analysts, instead of working directly with databases, work with datasets (data tables) using various batch solutions.

Backend developer (Kotlin)

At the same time, each industry has its own analysis standards, depending on the sensitivity of the data: for a government structure, the set of tools will be one (based on strict access restrictions), for non-profit organizations – another, for a digital startup – the third.
Where to get the data and what to do with SQL commands?

Organizations have their own data stores and can be accessed when needed. This happens both when differentiating user rights, and when working on the development of databases. Using Python, knowing the server address and the data for connecting to it, you can import the necessary libraries and write requests already inside the used program.
Examples of libraries: for ODBC – pyodbc, PostgreSQL – psycopg2, MySQL – mysql.connector, etc.

Your “recipient” program, in turn, can reside inside a cloud solution based on jupiter notebook. And you will apply further calculations to the obtained data and / or plot graphs (at least libraries: pandas, numpy, matplotlib, etc.).

Analytical functions

It is believed that in databases, each line of the query should be processed independently of the others. Practice dictates other tasks to us, in which it is often necessary to group rows and calculate common indicators for them that are used to assess the characteristics of rows within a group.

Examples:

calculate the frequency of communication of friends with each other (for example, it is necessary to determine the closest contacts within the selected circles): then we will look for the percentage ratios of the duration of negotiations, the frequency of correspondence and telephone conversations, the number of geographical intersections, visits to shared resources, etc.,
compare salaries of employees within departments, calculate ratings,
find out the top N customers in terms of services or products,
find out the workload of neighboring points of sale, etc.

SQL analytical functions come to the rescue – a powerful tool that helps unload the client from a large amount of procedural code, DBMS – from complex and sometimes ineffective queries, reduce development time and at the same time get the desired result.

The post Why data analysts need to know SQL appeared first on Terr For MER.