Data FAQ

APS Automatic Pattern Search - Data FAQ

Click here to get to the main FAQ page

Does APS come with a database of historical data?

What data format APS requires?

Is there any limit on the size of the historical data files?

What vendors provide data compatible with APS?

How to convert a Metastock data file to APS format?

What is the procedure to convert multiple Metastock files to APS format?

Can I use Tradestation data with APS?

How to convert TeleChart daily data to APS format?

Does APS interface with eSignal data?

What is the procedure for generating files in APS format with the Pinnacle data DMAKER?

How is intraday data used by APS?

Can I use System Tracking with Intraday data?

What type of futures data series is most appropriate to use?

Where can data files be saved?

How many bars should a file used in a search or scan contain?

How to convert QP3 data by Quotes Plus to APS Format?

Does APS come with a database of historical data?

APS does not include a database of historical data but only a few sample files to use with the demo in becoming familiar with the program operation. For a partial list of data vendor that provide data in APS compatible format click here.

What data format APS requires?

APS reads historical data file in "APS format". This format is a simple ASCII text of the form: YYYYMMDD OPEN HIGH LOW CLOSE per line of daily data or COUNT OPEN HIGH LOW CLOSE per line of intraday data. The data fields must be separated by a single space or comma. The format is explained in the program manual (help file that comes with the demo). You can also open the data files included with the program (use Wordpad) to see the proper format.

Is there any limit on the size of the historical data files?

There is no theoretical limit imposed by the program of the size of data files to use in the search or scan workspaces. However, the size of files affects the time takes to complete a search or scan. Practical limitations many include workstation memory and disk access speed but these constraints tend to become insignificant as new hardware technology is introduced.

What vendors provide data compatible with APS?

For examples of software that generates historical data files in APS format click here.

For a list of vendors see the Traders.com website. Click here.

How to convert Metastock data files to APS format?

There are two steps involved in using data from Metastock.

Step 1: You must first create a Metastock ASCII data file using the downloader
Step 2: Convert the Metastock ASCII file to APS format using the conversion Tools

Step 1: Go to Metastock downloader and select the file to convert. The source file is set to Metastock format and the target to ASCII. In the Destination part of the Convert Securities window of the Downloader select ASCII and for the fields to output mark all EXCEPT Time, Volume and Open Interest. Also mark the box to create new file. When done, click Apply and then OK. Then click OK to have the file converted.

Step 2: To convert Metastock ASCII to APS format

- From the main APS menu select Tools and then Data file Conversion.
- Select the source file and Metastock compatible for file settings. The file can be found in the directory you specified in the Downloader to store the converted files.
- In the target file settings leave the default APS format and hit Convert. Select a new the directory to store the APS format file and click Save to complete the conversion. You may use Windows Explorer to create a new directory.

For intraday files include the TIME field in Step 1 above when generating a Metastock ASCII file. The intraday ASCII file conversion to APS format procedure is:

- From the main APS menu select Tools and then Intraday File Creation.
- Select the source file and specify the correct file settings (mm/dd/yyyy and comma).
- Hit Convert and select a new directory to store the APS format file (or the same directory with a different file name) and click Save to complete the conversion.

Important note: Ticker names should not include spaces. If spaces are present the conversion will not work properly.

What is the procedure to convert multiple Metastock files to APS format?

Step 1: Convert Metastock format to Metastock ASCII format

- Go to Metastock Downloader, select Tools and then Convert. In the Convert Securities window set the following: For the Source part select Metastock format and then click Browse and select the directory of interest. Highlight the first file in the directory and then hold down the SHIFT key on your keyboard and highlight the last file. That should cause all files in the directory to be highlighted. Mark the By Symbol option and then click Open. In the Destination part of the Convert Securities window select ASCII for file type and MANUALLY input the path in the folder field of the directory where you want the converted files to be saved at. (ex. C:\Spdata ). Then, input *.txt in the file name field. DO NOT use Browse for this step.

- In the same window, click Options and select Destination. Mark ONLY Replace existing files and Create new files options. For the fields to output mark all EXCEPT Time, Volume and Open Interest. When done, click Apply and then OK. Then click OK to have the files converted.

Step 2: Convert Metastock ASCII to APS format

- From the main APS menu select Tools and then Metastock ASCII Conversion.
- In the Metastock ASCII text source directory select the directory you specified in the Downloader to store the converted files (ex. C:\APSDATA).). The files should appear in the window list. You may mark those you like to be converted or mark the Select all option for converting all files.

- In the APS ASCII text target directory select the directory to store the APS format files. You may use Windows Explorer to create a new directory, (ex. c:\APSDATA).

- Click Convert to start the conversion.

Can I use Tradestation data with APS?

Data from Tradestation can be used with APS after converted to APS format.

Tradestation 8.1: S ave the historical data in a .txt file using the following command sequence:

View, data window, show all, save

The format of the file may not be compatible with APS (see help file for format specifications) but you can use the File Conversion option from the Tools menu of APS to convert the file to the proper format (see below).

Tradestation 5 or below:

For daily data:

Step 1: Create a text file using TS data server by outputting the fields DATE, OPEN, HIGH, CLOSE, LOW. Then, use your editor, for example, Wordpad, to determine the exact format of the file generated by TS and remove any data header if present.

Step 2: Convert the TS text file to APS format using the conversion Tools.

- From the main APS menu select Tools and then Data file Conversion.
- Select the source file and specify the correct file settings.
- In the target file settings leave the default APS format and hit Convert. Select a new directory to store the APS format file (or the same directory with a different file name) and click Save to complete the conversion.

For intraday data:

Step 1: Use the TS data server to generate an ASCII intraday text file with the option to output the following fields only: Date, time, open, high, low, close. Then, remove the file header using your editor (the first line in the file with the field names) and then use the Intraday File Creation option from the Tools menu to convert that to an intraday file. In that case, the date will be replaced by an increasing integer number and the time will be omitted.

Step 2: Convert the TS text file to APS format using the conversion Tools.

How to convert TeleChart daily data to APS format?

In order to generate data for APS from TeleChart, follow the procedure below:

1. Create a watchlist with the symbols you like to export or use a TeleChart list, like NASDAQ 100, for example
2. Go to Databank | Export to Text
3. Choose the Watchlist to export.
4. Choose the data fields to export. They must be DATE in YYYYMMDD format, OPEN,HIGH, LOW and CLOSE, in that order.
5. The delimiter must be set to "space"
6. Choose to export each each ticker in its own file.
7. Specify the directory to export the data. You may use the default
C:\Program Files\APS\data or any other directory you like. (ex. C:\stocks)
8. Choose Ascending order
9. Choose the number of days to esxport (anything above 2500 is recommended).
10. Click on Export.

Does APS interface with eSignal data?

APS doesn't interface directly with eSignal but there is a small program Qcollector, very inexpensive, that converts eSignal data to APS compatible format. You can use that with both daily and intraday data. To download a demo please visit: http://www.mechtrading.com/

For daily data:

In the options of Q-Collector specify to output the fields Date, Open, High, Low, Close. The fields can be either comma or single space delimited and for the date use the format YYYYMMDD. The output file can be used by APS directly.

For intraday data.

In the options of Q-Collector specify to output the fields Date, Time, Open, High, Low, Close. The fields can be either comma or single space delimited and for the date use the format YYYYMMDD. The time field must have the format hh:mm. The output file cannot be used by APS directly. You must use the "Intraday File Creation" option from Tools menu to convert the file to APS intraday format (See: http://www.tradingpatterns.com/Software/APS/intraday/intraday.html)

In both cases above, daily or intraday data, there should be no header in the data file ( The first line of the file is often a header that defines the content of the fields in the records that follow. QCollector gives you the option of whether or not to include an ASCII header in your data files.)

What is the procedure for generating files in APS format with the Pinnacle data DMAKER?

DMAKER uses a set of files output the data in text or Metastock files. Those files start with an "$" sign and there are some default files available with the program, one is called $CLCLONG.asc.You have the choice of either creating your own custom file, (ex. $custom.asc), to include just the contracts you follow, or modify the existing $CLCLONG.asc to create ASCII files in APS format. You only have to specify the conversion parameters in the first contract as in the following example:

\DATA\CUSTOM\BP.ASC
#STARTING DATE = 01/01/1990
#ENDING DATE = 12/31/2049
#OUTPUT DATE FORMAT = 3
#OUTPUT COLUMN HEADINGS = N
#COL HEADINGS IN QUOTES = N
#INCLUDE HOLIDAYS = N
#VALUE OF INCLUDED HOLIDAY = P
#COL SEPARATOR CHARACTER = S
#SAMPLING PERIOD = D
#OVERWRITE OUTPUT FILE = Y
#COT INTERPOLATION METHOD = 2
#COT INDEX LOOKBACK DAYS = 400
#FIRST LINE TITLE =
#CLC LINKING METHOD = 3
#CLC NEG PRICE HANDLING = 5
#IDX INTERPOLATION METHOD = 2
#IDX INCLUDE SAT TRADING = Y
#OUTPUT FORMAT = A
BP_CLC 1,2,3,4,6,7

\DATA\CUSTOM\CC.ASC
CC_CLC 1,2,3,4,6,7

\DATA\CUSTOM\KC.ASC
KC_CLC 1,2,3,4,6,7

\DATA\CUSTOM\HG.ASC
HG_CLC 1,2,3,4,6,7

\DATA\CUSTOM\C_.ASC
C__CLC 1,2,3,4,6,7

etc.

---------------------------

You may specify the starting date according to your requirements. Please note that in the past contracts that started after the starting date had zeros outputted for all dates before that. This can be corrected by creating a different $...asc files for those contracts. Pinnacle data will update your ASCII files after each download. Just as a reminder, make sure to run DMAKER often to take care of contract rollovers. To add a new $file to dmaker, go to DOS and in the pinnacle directory and you will find a file called dmlist.dat. Edit that file and after the last line in the *CLC section add the name of the new file and then under that the file name (without the ASC extension).

How is intraday data used by APS?

Intraday data is used in APS exactly as daily data. The program does not consider time in its algorithm and this speeds the execution time significantly. In order to use intraday data with APS, an ASCII text file must be created, in which the time field is omitted and the date field is replaced by and increasing 8-digit integer: Example:

10000001 1492.25 1493.50 1486.75 1489.25
10000002 1489.00 1489.25 1482.50 1485.50
10000003 1485.75 1488.00 1484.00 1484.75

For more information see: http://www.tradingpatterns.com/Software/APS/intraday/intraday.html

Can I use System Tracking with Intraday data?

We recommend against using System Tracking to generate signals for intraday patterns. Theoretically, if one can update intraday data files in APS format and then run the conversion using the Tools menu, any models saved in System Tracking will update also with new signals, if any. It is preferable though to generate code for intraday patterns and program then into strategies in either TS or Metastock with a live data feed.

What type of futures data series is most appropriate to use?

Use continuous, backwards-adjusted series for futures and split adjusted series for stock data.

Where can data files be saved?

Data for APS, in APS format, can be stored in any directory. It is recommended to keep data files in APS format separate from original files in other formats so the error checking routines in Tools can be used.

How many bars should a file used in a search or scan contain?

At least five years of daily data and 15,000 bars of intraday data is recommended for discovering statistically significant patterns with APS. The more data that is available, the better it is in principle.

How to convert QP3 data by Quotes Plus to APS Format?

1. Run QP3 (c:\program files\Qotes Plus\Dispay) to get the main program chart display.
2. From the main menu select: Options and then Search for Symbol (or just type "N')
3. Select the files you would like to put in a list or lists. Make sure you specify in "continuous contracts" for commodities and the appropriate linking method when selecting the symbols from the list (volume 1 is recommended).
4. After the list is created from the main menu select Tools and then ASCII Export.
5. Create the new format (this step must be done just once) [Date(CCYYMMDD), Open, High, Low, Close)] and save it.
6. Click Export to ASCII and specify the name of the list of symbols to export and the target directory to save the files. The file extension must be set to TXT and yu must mark "Send to individual files" in Export Type option.
7. Choose the ASCII format and then click Export.

Since the exported format by QP3 is not exactly the format required by APS, you can use the File Conversion option from the Tools menu of APS to convert each file exported by QP3 to APS format. (Note: starting with version 4.9 upgrade, a conversion to APS format is not required, as this version can read comma delimited data generated by QP3).