Creation of a Neural Network Algorithm for Automated Collection and Analysis of Statistics of Exchange Quotes Graphics

Currently, the problem of automated data analysis and statistics collection from stock quotation charts has not been fully resolved. Most of the analysis of visual data falls on the physical work of the analyst, or on obsolete software solutions. The process of summarizing the information received from financial markets still requires physical attention and labor, which increases the risks associated primarily with the human factor and corresponding errors. An algorithm has been developed and tested for the automated collection of statistics from graphs of stock quotes, including data on the development and context of various figures (patterns) of technical analysis, as well as an improved adaptation and tracking system for the trend. The modeling process, analysis and the results of applying the analysis algorithm and statistics collection are presented. The developed algorithm works in conjunction with the previously created neural network pattern detector, which allows to automatically search for the exact boundaries of technical analysis figures of various sizes, analyze the context in front of them and play the patterns. This makes it possible to obtain important statistics that allow one to determine the degree of confidence in emerging patterns, taking into account their type, context, and other factors. In terms of accuracy and efficiency, the developed algorithm meets the existing challenges in the financial markets and can significantly increase the efficiency of the trader or investor through the automated processing of graphic and visual data. The created solution is universal in nature and can be applied to any capital market, regardless of the location and nature of the assets placed. The results can be used both to improve the accuracy of existing trading strategies, and for the analytical work of financial market participants. The use of new technologies for statistical processing of information can significantly improve the accuracy of investment and trade decisions.


Introduction
To work effectively in financial markets, a trader or analyst needs to be able to read the mood of investors, evaluate the current market trend, and also work with statistics. For these purposes, trend direction determination, fundamental indicators and technical indicators are used. A trend is a general direction in which a stock price moves. Depending on whether the market is bullish or bearish, trends move up or down, respectively. Stock market trend analysis is an aspect of technical analysis that attempts to predict future stock movements based on historical data. The stock market trend is based on the concept that past movements are windows for future trends. While fundamental indicators, with the exception of force majeure circumstances, are weakly variable and are interpreted rather unambiguously, then the trend line and technical indicators are more dynamic and often do not have an unambiguous interpretation. Moreover, in contrast to the fundamental values, the technical analysis of patterns (figures) currently does not have accurate performance statistics in relation to each traded quote (stock) on the exchange. It is important to note that if the trader is tasked with collecting such statistics, then several hundred hours of physical labor for each quote will be spent on its solution. The lack of data on a traded financial instrument, regardless of the popularity and capital intensity of the exchange, negatively affects decisions due to the limited physical resources of a person and the resulting "human factor" error. From the above it follows that today the problem of investors in the capital market consists not only in the manual mode of visual analysis, but also in the absence of standardized statistics on the effectiveness of patterns, including market conditions before their formation -the context. This scientific work is a continuation of the article [1], which describes the use of a convolutional neural network to create a detector of figures of technical analysis on graphs of stock quotes. The algorithm developed as part of the current scientific work automatically adapts the parameters of the trend lines to patterns of various sizes, which, combined with the detection of technical analysis patterns, allows to extract unique statistical data from stock quotation charts. After determining the boundaries and the type of pattern and trend, it becomes possible to automatically consider the context of the ongoing price movements: determine the market trend preceding the patterns, the percentage of successfully worked out figures, analyze volumes, etc. The obtained exchange data can be used for analytical research and building new strategies on capital markets, as well as improve the accuracy of existing ones.

Literature review and problem statement
The use of artificial intelligence as a technology of statistical data processing has a fairly wide representation in scientific papers. In [2], the authors develop a forecasting system for trading solutions that combines technical analysis with computational intelligent methods. The forecasting problem is formulated as a classification problem with three class values, which are buy, hold and sell signals, and the generation of a trading signal follows from a trend analysis. In [3], an adaptive system for making trading decisions on stock indices to predict the movement of stock prices is presented. In [2,3], the authors use a hybrid neural network for the purpose of making decisions and forecasting prices, relying on standard technical indicators. In [4], the authors conduct a statistical analysis of stock prices of a number of companies in the construction In the course of the study, the authors conclude that the SMA most adequately reflects the general trend in the value of stocks of companies, as well as the direction of the trend. Works [2][3][4] share common shortcomings: a small number of indicators and their lack of universality, due to which it is impossible to predict the behavior of stock prices under unusual conditions. In [5], the authors conduct a study of the effectiveness of automated trading based on the strategy of applying SMA on shares of BRICS countries. To conduct this study, a comprehensive portfolio of assets traded in the markets of each BRICS member is created. In the course of work on several selected portfolios, the results are higher than those of the rather popular buy-and-hold strategy. It is worth noting that, despite local successes, on the whole, rather heterogeneous results are recorded, and a relatively short time period is also chosen. In [6], the author examines 281 double bottom figures in 500 companies and the S&P 500 index chart. The study covers the period from 1991-1996 and 2000-2004 years. The study found that most often the model reaches 35 % growth from the target price, and in 28 % of the considered cases, 45 % growth is achieved. It is important to note that the data covered a relatively short period of time, statistics are relevant on timeframes of at least 1 day, and attention is not focused on statistics on specific companies traded on the exchange. In [7], the authors develop methods for forecasting financial time series using deep learning in relation to the FOREX market. In the research paper [8], deep learning algorithms are used to predict the behavior of financial markets, the research includes stocks included in the S&P 500 index. In [7] and [8], the authors use indicators that aggregate data to predict prices, rather than building an expert system. The authors in [9] use artificial intelligence for short-term forecasting of stock prices in the stock market. In the article, the authors examine direct distribution neural networks and recurrent neural networks. The study found that for the task, a multilayer perceptron is more effective than a recurrent network. In [10], an algorithm is developed for forecasting exchange rates and trade using expert information processing techniques, namely, the Delphi method. The study uses the network architecture of LSTM (Long Short-Term Memory). The algorithm is tested at exchange rates, but the question remains how it will behave in other financial markets, such as stock and commodity markets. In [11], a neural network stock trading system is used, based on the use of evolutionarily optimized parameters of technical analysis. In the study, genetic algorithms are used to create points of purchase and sale, and stocks included in the DJ 30 index are selected as a training sample. In [12], the authors explore the possibility of forecasting prices on the FOREX market using deep learning on EUR/USD, GBP/USD, JPY/USD trading pairs, and in [13] they predict the value of shares using a machine algorithm, based on data from January 2015 to June 2018. In [14], the authors investigate seasonal fluctuations in the currencies of the euro and the renminbi based on data from the World Bank using a multilayer perceptron network. During the experiments, it is found that forecasting the exchange rate of two currencies can be predicted on the basis of statistical, causal and intuitive methods. In [15], as in [14], researchers study the possibility of forecasting exchange rates, and satisfactory results are obtained during the study, but the system needs additional study, as well as testing on a large amount of data. In [16], signals from a number of technical indicators are used as input data, and machine learning methods and statistical methods are used as a data processing tool. The study found that the used neural network system does not show stable profits. In [17], a review of big data media is carried out, as well as decision-making methods using such data. Various methods of analytics and modeling based on the use of big data are considered. The study itself is of a review nature, but thanks to competent generalization, it is possible to identify important relationships. In [18], an effective intelligent forecasting system is introduced to increase the accuracy of existing systems. The proposed system includes adaptive filtering, artificial neural networks and evolutionary optimization. The developed hybrid system for predicting price changes based on the S&P 500 data has proved to be quite effective, but needs further verification on a more expanded data volume. In [19], a fuzzy logic system is used to create an index that could reflect the importance of the size of a financial institution, its interconnectedness and irremovability in the Colombian financial market. In addition, another index is created using an analysis of the main components, and both are used to find relevant information when examining the systemic importance of financial institutions in Colombia. The result of the work is local in nature, but can be extrapolated to work with a trend in capital markets. The aim of research is to create an automated algorithm for collecting and analyzing statistics of graphs of stock quotes, using data on the boundaries of technical analysis figures obtained by the figure detector presented in [1].
To achieve the aim, the following objectives are set: -development of a mathematical tool that allows to automatically determine the trend line for figures of various durations, adapting to the size of the figure; -creation of an algorithm for automated determination of the context and testing of detected patterns; -evaluation of the work of the created algorithm for collecting and analyzing statistics of graphs of stock quotes.

Materials and working methods of the algorithm for collecting and analyzing statistics of graphs of stock quotes 1. The principle of adaptive moving average
Since the developed algorithm processes the data obtained as a result of the operation of the detector of figures of technical analysis presented in [1], let's briefly explain the principle of operation of the detector, which is shown in Fig. 1.
The input for the pattern detector is presented as a column vector: The algorithm of the pattern detector located on the charts of financial markets is as follows: 1) according to financial markets for a selected time period, a "window" of various durations is run, breaking it into frames; 2) the frames highlighted by the "window" are normalized to the range from 0 to 1 according to the formula (2): where f() -the normalization function, p -the numerical value of the discrete signal highlighted by the "window", x -the column vector of the values highlighted by the "window"; 3) the received data is scaled and fed into a convolutional neural network to determine the probability of frames belonging to one of the types of patterns or noise.
Based on the literature analysis, a simple moving average method is chosen to determine the trend, which is an algorithm for smoothing time series in order to exclude the influence of a random or noise component [20]. The method consists in replacing the values of the members of the series with the arithmetic mean of the corresponding window: where n -the window size (smoothing period), m -the number of a member of the series whose value is replaced by the average. The main problem when using the moving average method is the uncertainty in choosing the size of the window. The top of Fig. 2 shows a graph of stock quotes, on which the detector highlighted three "double bottom" figures of various sizes, as well as moving average graphs with window values adapted for figures of different durations (green, red and purple).  To compensate for the delay in signal processing, a dynamically changing window size is used for the first and last window values.

2. Algorithm for automated determination of the context and refinement of detected patterns
After determining the trend lines adapted for each individual figure, it becomes possible to analyze and collect statistics that are not tied to the duration of the figures.
Based on the analysis of the literature, the parameters for determining the trend in front of the figure: -downward: each previous value of the trend graph is greater than the current one. Total growth of at least 10 % of the amplitude size of the figure; -upward: each previous value of the trend graph is less than the current one. The total drop is not less than 10 % of the amplitude size of the figure; -lateral: previous amplitude values of the trend vary within 10 % of the amplitude size of the figure for at least half of the duration of the figure.
The development of the figure is determined from its right border. The figure is marked as fulfilled with a positive result if the amplitude value of the price chart increased by at least 30 % of the amplitude size of the figure, while not dropping more than 35 % of the value at the end of the figure (to take into account a possible rollback of the price).
The parameters for determining the trend and working out are customizable and can be changed at the request of the analyst or trader with recalculation of the received statistics.

The results of the algorithm for collecting and analyzing statistics of graphs of stock quotes
The result of the algorithm is presented in the lower part of Fig. 2. The developed algorithm makes it possible to automate the receipt of important statistics of graphs of stock quotes -the direction of the trend before the pattern (context) and the development of patterns. Application of the developed method for adapting the moving average window allows to evaluate the obtained statistical data regardless of the duration of the patterns, making the statistics relevant for use with graphs with any timeframes.
The algorithm was tested on real data from stock charts, where it allowed to obtain relevant statistics, the analysis of which will be devoted to a separate scientific work. The data processing for 1000 days by the presented algorithm takes about 0.03 seconds, and the total processing time together with the neural network pattern detector was about 0.68 seconds on a personal computer in the configuration of Core i7-8700, 32 GB of RAM, NVIDIA GeForce GTX 1080.

Results of studies on the creation of an algorithm for the automated collection and analysis of statistics on graphs of stock quotes
As a result of the studies, an algorithm was developed that automates the receipt of statistical data on the detected patterns of technical analysis of various sizes on the charts of stock quotes.
The developed algorithm allows to select and standardize the following statistics: 1) determining the direction of the trend in front of the figure (context), its duration and amplitude values. The context preceding the appearance of patterns has a serious effect on them, and considering a pattern does not make sense without knowledge of its context; 2) determining the fact of winning back the figure, its value relative to the amplitude size of the figure and the time period for its achievement. By analyzing the price values after the pattern, it is possible to determine the percentage of successfully worked out figures that were on the chart earlier in order to assess the degree of confidence for each type of chart for a particular chart or their combination.

Computer Sciences
By changing the conditions for calculating statistics, a trader or analyst can get unique statistical data of interest to them both for individual quotes and for their totality for any timeframes. The speed of the algorithm allows to analyze the statistics of patterns in real time.

Discussion of the results of the neural network algorithm for the automated collection and analysis of statistics on graphs of stock quotes
Currently, when analyzing stock charts, investors have access to a set of fundamental indicators, the latest news reports and various technical indicators. Effective and successful results in capital markets consist not only of the possession of the above working tools, but also of the correct assessment of the incoming information, as well as the ability to ignore information noise and highlight the most important signals. In conditions of high information flow, especially in force majeure circumstances, information noise and the flow of false signals can increase significantly, which, given the limited human resources, has serious negative consequences.
In such conditions, the question of the use of software products for statistical data processing is particularly acute, because how the use of such systems allows to process more data, filter out information noise, and also significantly reduces the likelihood of an error associated with the human factor. In terms of accuracy and efficiency, the developed algorithm for the automated collection and analysis of statistics on graphs of exchange quotes meets the existing challenges in the financial markets, and is also able to significantly increase the investor's efficiency by automated processing of graphic and visual data. It is important to note that the decision is universal in nature and can be applied to any capital market, regardless of the location and nature of the assets placed on it. The developed algorithm, coupled with the previously created pattern detector, is able not only to replace the eyes of the trader and detect a particular pattern, but also to consider the context of the appearance of the pattern, as well as provide unique statistics for the selected period of time, which is not currently available to the vast majority of similar technical decisions. Based on the study, it is planned to further develop this area and expand the range of innovative tools for analyzing stock quotes charts that can be used.

Conclusions
The developed method of adapting the moving average window for figures of various durations allows standardizing the obtained statistical data when analyzing graphs with any timeframes.
The developed algorithm allows to automate the determination of the direction of the trend in front of the pattern (context) and the development of patterns, obtaining an estimate of their duration in time and amplitude values. The parameters for determining the trend and working out the patterns are customizable and can be changed to the requirements of the analyst or trader with recalculation of the received statistics.
The data processing for 1000 days by the presented algorithm takes about 0.03 seconds, and the total processing time together with the neural network pattern detector is about 0.68 seconds and will be reduced in the future.
The pattern detector, coupled with the developed algorithm for collecting and analyzing statistics of graphs of stock quotes, allows to automatically search for the exact boundaries of technical analysis figures of various sizes, analyze the context in front of them and wager. This makes it possible to obtain important statistics that will determine the degree of confidence in emerging patterns, taking into account their type, context and other factors, for example, volumes.