Discrete wavelet transform using USD-CNY exchange daily time series dataset
We have discussed applications and the basic concepts of wavelet transform, general process and signal extension modes of Discrete Wavelet Transform (DWT) and methods of 1D one stage DWT, and many other topics on Wavelet transform. After you have all this basic knowledge, we move on to a real-world project on single-Level DWT of 1D USD to CNY exchange time series dataset.
1. Introduction to the Project
The objective of this project is to make single-level Discrete wavelet transform (DWT) on a 1D time series signal, which includes the following main tasks:
- Make single-level decomposition
- Visualize the Approximation and Detail coefficients
- Reconstruct the time series signal
- Reconstruct the Approximation and Detail
- Visualize the Approximation and Detail
- Make data noise removal, or noise reduction, or subtraction
2. Prepare Data
(1) Import required packages
Pandas to read the dataset and display it in DataFrame, “PyWavelet
matplotlib for visualization.
import pandas as pd import matplotlib.pyplot as plt import pywt
(2) Read the data
We use historical USD to CNY exchange daily dataset during September 24, 2012 to September 24, 2022. If you use this dataset for other things more than personal study, please cite the dataset source: ca.investing.com, where I downloaded the dataset. For your convenience, I put it in my GitHub repository. You can read the dataset directly from my GitHub repository. If you are not familiar with the method to read dataset from GitHub using
Pandas, please read this post.
url = 'https://raw.githubusercontent.com/shoukewei/data/main/data-wpt/USD_CNY%20Historical%20Data.csv' df = pd.read_csv(url) df.head()
(3) Slice the time and price signal
Suppose we use the
Price column as the signal (S) and
Data column as the time axis of the signal (t).
t = pd.to_datetime(df["Date"]) S = df['Price']
(4) Plot the signal
Now, let’s visualize the signal using
fig, ax = plt.subplots(figsize=(15,5)) ax.plot(t,S) ax.set_xlabel('t (Day)',fontsize=14) ax.set_ylabel('Signal (Price)',fontsize=14)
3. Single-level Wavelet Decomposition
In this section, we decompose the signal at the first level using DWT methods, which were discussed in previous posts,
(1) 1D single-level DWT
db2 wavelet and
sym signal extension model in this project.
(cA,cD) = pywt.dwt(S,'db2','sym')
(2) Print the coefficients
print('Approximation Coefficient: \n',cA) print('Detail Coefficient: \n',cD)
The results look as follows:
Approximation Coefficient: [10.06329622 10.03529818 9.9410734 ... 8.90362945 8.9165788 8.92294747] Detail Coefficient: [ 0.03080233 -0.00255284 0.00374649 ... 0.00016898 0.00163954 0.00093243]
(3) Length of the coefficients
We can print the coefficient length using the following method, or the method introduced in the previous post.
print('Length of orignal signal: ',len(s)) print('Length of Approximation Coefficient: ', len(cA)) print('Length of Detail Coefficient: ',len(cD))
The results are:
Length of orignal signal: 2593 Length of Approximation Coefficient: 1298 Length of Detail Coefficient: 1298
4. Visualization of Approximation and Detail Coefficients
We plot the approximation and detail coefficient, and then save the plot in the result folder in the working directory. If you are interested in systematic methods on data visualization, you can start from this post.
fig,axs = plt.subplots(1,2,figsize=(15,5)) plt.rc('xtick',labelsize=15) plt.rc('ytick',labelsize=15) axs.plot(cA) axs.set_ylabel('1st level Approximation coefficient (cA)',fontsize=15) axs.set_xlabel('t (year)',fontsize=15) axs.plot(cD) axs.set_ylabel('1st level Detail coefficient (cD)',fontsize=15) axs.set_xlabel('t (year)',fontsize=15) plt.tight_layout() plt.savefig('./results/approx_detail_coeff_1level.png',dpi=500) plt.show()
5. Reconstruct the Signal
Here, we use the
idwt method to reconstruct the signal.
(1) inverse discrete wavelet transform
S_rec = pywt.idwt(cA,cD,'db2','sym')
(2) Check the lengths
Let’s check if the length of reconstructed signal is the same with original signal. We have discussed that odd length signal will have one more extra value added at the end during DWT in the previous post.
print('Length of orignial signal: ',len(S)) print('Length of reconstructed signal: ',len(S_rec))
The results are:
Length of orignial signal: 2593 Length of reconstructed signal: 2594
(3) Reconstructed signal
As we discussed in the method posts, the final reconstructed signal is the one after removing the last value.
S_rec = S_rec[:-1]
(3) Evaluate the reconstruction
First, we check how well the reconstructed signal presents the original one using visualization method.
# a simple plot plt.plot(S) plt.plot(S_rec)
It seems that we cannot the different between the constructed signal and original one. Next, let’s check the reconstruction performance by the Mean absolute error (MAE) and maximum absolute error (Max. AE).
# errors print('MAE:',sum(abs(S-S_rec))/len(S)) print('Max. AE:'max(abs(S-S_rec)))
The results are:
MAE: 7.463712983133717e-16 Max. AE: 2.6645352591003757e-15
There are very small errors, which can be ignored. It confirms that DWT can well reconstruct the original signal.
6. Partial Wavelet Reconstruction
(1) Reconstruct the approximation and detail
In the method post, we introduced two methods to reconstruct the level 1 approximation and detail (A and D) from the coefficients cA and cD. In this project, we use the
idwt method. As we discussed in the method post and previous section, our signal has odd length, so we should remove the last values.
A = pywt.idwt(cA, None,'db2','sym') D = pywt.idwt(None, cD, 'db2','sym') A = A[:-1] D = D[:-1]
(2) Visualization of the approximation and details
fig,(ax1,ax2,ax3) = plt.subplots(3,1,figsize=(15,10)) plt.rc('xtick',labelsize=15) plt.rc('ytick',labelsize=15) ax1.plot(t,S) ax1.set_ylabel('Original price signal (S)',fontsize=15) ax2.plot(t,A) ax2.set_ylabel('Approximation of Signal Price(A)',fontsize=15) ax3.plot(t,D) ax3.set_ylabel('Detail of Signal Price (D))',fontsize=15) ax3.set_xlabel('t (year)',fontsize=15) plt.tight_layout() plt.savefig('./results/approx_detail_1level.png',dpi=500) plt.show()
7. Noise Reduction
Detail constituent is the high frequency data, which can be removed as noise. There are some other detailed threshold methods to remove the noise. We just remove the 1st level detail as the noisy in this project. In another way, we just keep the approximation constituent to represent the original signal. Let’s check how well it is.
(1) Visualize approximation vs. original
We visualize the denoised signal (i.e. Approximation in this project) and the orignal signal.
fig,ax = plt.subplots(figsize=(15,6)) plt.rc('xtick',labelsize=15) plt.rc('ytick',labelsize=15) ax.plot(t,S) ax.plot(t,A) ax.set_ylabel('USD to CNY Exchange Price',fontsize=15) ax.set_xlabel('t (Daily)',fontsize=15) ax.legend(('Orignal price signal (s)','Approximation (A)'),loc='best',shadow=True) plt.savefig('./results/approx_orignal.png',dpi=500) plt.show()
Visualization comparison result illustrates that the approximation can well represent the original signal.
(2) Check errors
Let’s check the erros.
err = sum(abs(S-A))/len(S) max_err = max(abs(S-A)) print('MAE:',err) print('Max. AE:',max_err)
The error statistics are:
MAE: 0.004110424877766035 Max. AE: 0.048823329323957054
The errors are very small, which also conforms that the approximation can well represent the original signal.
However, from the denoised point of view, the denoised signal (approximation) is still not so smooth because we only remove the first level details in this project. One-stage DWT is a quick method to make DWT for a signal, but it is suitable for the signal with low frequency. For a high-frequency signal or dataset, a multi-level DWT is needed, and we will talk about this topic in the future.
If you are interested in learning how to apply wavelet transform to real-world cases with Python step by step, you are welcome to visit our online school to enroll my courses:
(1) Very basic one
Practical Python Wavelet Transforms (I): Fundamentals
(2) For Real-world projects