Backtrader量化平台教程

我们知道，有时候我们要测试的可不仅仅是一个标的，但是之前backtrader的教程中，我们都是针对的是一个标的的回测。如果接触过优矿、ricequant这样的平台的同学，可能觉得backtrader不适合做这样的portfolio层面的回测。确实，似乎backtrader整个官方教程里面，没有任何讲到这种全市场、组合的回测demo，但是backtrader其实也是可以胜任这样的任务的。

前段时间，笔者就做了这样的一个事情，让backtrader能够完成我们想要的组合层面的回测。

1.最终的效果

和一般的portfolio层面的回测平台一样，我们希望，最后我们实现一个策略只要进行一些设置就可以了。笔者利用backtrader封装了一个函数，实现了几乎和优矿一样的功能。

使用的时候，笔者的函数只需要如下的设置：

[python]

1. start_date="2017-04-01"
2. end_date="2017-06-20"
3. trading_csv_name='trading_data_two_year.csv'
4. portfolio_csv_name='port_two_year.csv'
5. benchmark_csv_name=None

然后我们就可以回测了。笔者把回测的类封装了起来，只要调用笔者的回测类就可以了。
[python]

1. begin=datetime.datetime.now()
2. result_dict=bt_backtest.backtrader_backtest(start_date=start_date,end_date=end_date,trading_csv_name=trading_csv_name,\
3. portfolio_csv_name=portfolio_csv_name,bechmark_csv_name=benchmark_csv_name)

然后，回测结束后输出评价指标。
[python]

1. end=datetime.datetime.now()
2. print"timeelapse:",(end-begin)
3. print'StartPortfolioValue:%.2f'%result_dict['start_cash']
4. print'FinalPortfolioValue:%.2f'%result_dict['final_value']
5. print'TotalReturn:',result_dict['total_return']
6. print'SharpeRatio:',result_dict['sharpe_ratio']#*2#todothereshouldbeconsider!
7. print'MaxDrowdown:',result_dict['max_drowdown']*2
8. print'MaxDrowdownMoney:',result_dict['max_drowdown_money']
9. print"TradeInformation",result_dict['trade_info']
11. result=pd.read_csv('result.csv',index_col=0)
12. result.plot()
13. plt.show()
14. position_info=pd.read_csv('position_info.csv',index_col=0)

接下来说一下我们要输入的这些csv文件吧。
[python]

1. trading_csv_name='trading_data_two_year.csv'
2. portfolio_csv_name='port_two_year.csv'
3. benchmark_csv_name=None

第一个是交易行情的数据，数据格式如下：

tradingdate,ticker,_open,_high,_low,_close,_volume

20070104,000001,14.65,15.32,13.83,14.11,69207082.0

20070104,000002,15.7,16.56,15.28,15.48,75119519.0

20070104,000004,4.12,4.12,3.99,4.06,1262915.0

20070104,000005,2.51,2.53,2.46,2.47,14123749.0

20070104,000006,13.5,14.07,13.39,13.7,15026054.0

20070104,000007,2.44,2.44,2.35,2.36,3014956.0

20070104,000008,4.16,4.24,4.15,4.16,818282.0

20070104,000009,0.0,0.0,0.0,4.23,0.0

20070104,000010,0.0,0.0,0.0,5.37,0.0

20070104,000011,5.78,5.85,5.42,5.45,4292318.0

20070104,000012,11.15,11.3,10.66,10.95,6934903.0

所以，这一部分的数据会相当的大，一天就会有3000条左右的记录，毕竟，A股的股票数目就是这样。如果是用的更小级别的数据，那么数据量必然更大了。

[python]

1. portfolio_csv_name='port_two_year.csv'

这是我们调仓日的目标仓位，只需要sigdate,secucode,weight三个字段就行。

benchmark_csv_name自然就是benchmark的daily return数据文件了。这里可以不设置。

2.回测函数

接下来就是核心的回测的函数了。

[python]

1. #-*-coding:utf-8-*-
3. from__future__import(absolute_import,division,print_function,
4. unicode_literals)
6. importdatetime
7. importbacktraderasbt
8. frombacktraderimportOrder
9. importpandasaspd
10. importmatplotlib.pyplotasplt
13. #CreateaStratey
14. classAlphaPortfolioStrategy(bt.Strategy):
16. deflog(self,txt,dt=None):
17. dt=dtorself.datas[0].datetime.date(0)
18. print('%s,%s'%(dt.isoformat(),txt))
20. def__init__(self,dataId_to_secId_dict,secId_to_dataId_dict,adj_df,end_date,backtesting_length=None,benchmark=None,result_csv_name='result'):
21. #1.gettargetportfolioweight
22. self.adj_df=adj_df
23. self.backtesting_length=backtesting_length
24. self.end_date=end_date
25. #2.backtraderdata_idandsecIdtransferdiction
26. self.dataId_to_secId_dict=dataId_to_secId_dict
27. self.secId_to_dataId_dict=secId_to_dataId_dict
28. self.benchmark=benchmark
29. #3.storetheuntradabledayduetotheupanddownfloor,emptyitinanewadjustmentday
30. self.order_line_dict={}
31. self.pre_position_data_id=list()
32. self.value_for_plot={}
33. #4.keeptrackofpendingordersandbuyprice/commission
34. self.order=None
35. self.buyprice=None
36. self.buycomm=None
37. self.value=10000000.0
38. self.cash=10000000.0
40. self.positions_info=open('position_info.csv','wb')
41. self.positions_info.write('date,bt_id,sec_code,size,lastprice\n')
43. defstart(self):
44. print("theworldcallme!")
46. defnotify_fund(self,cash,value,fundvalue,shares):
47. #updatethemarketvalueeveryday
48. #print("actualvalue:",value)
49. self.value=value-10000000.0#wegivethebrokermore10millionforthepurposeofilliquidity
50. #self.value=value#wegivethebrokermore10millionforthepurposeofilliquidity
52. self.value_for_plot[self.datetime.datetime()]=self.value/10000000.0
53. self.cash=cash
54. #print("cash:",cash)
56. defnotify_order(self,order):
57. iforder.statusin[order.Submitted,order.Accepted]:
58. #Buy/Sellordersubmitted/acceptedto/bybroker-Nothingtodo
59. return
60. #Checkifanorderhasbeencompleted
61. #Attention:brokercouldrejectorderifnotenougthcash
62. iforder.statusin[order.Completed]:
63. iforder.isbuy():
64. #order.
65. self.log(
66. 'BUYEXECUTED,Price:%.2f,Cost:%.2f,Comm%.2f'%
67. (order.executed.price,
68. order.executed.value,
69. order.executed.comm))
70. self.buyprice=order.executed.price
71. self.buycomm=order.executed.comm
72. else:#Sell
73. self.log('SELLEXECUTED,Price:%.2f,Cost:%.2f,Comm%.2f'%
74. (order.executed.price,
75. order.executed.value,
76. order.executed.comm))
77. self.bar_executed=len(self)
78. eliforder.statusin[order.Canceled,order.Rejected]:
79. self.log('OrderCanceled/Rejected')
80. eliforder.status==order.Margin:
81. self.log('OrderMargin')
82. self.order=None
84. defnotify_trade(self,trade):
85. ifnottrade.isclosed:
86. return
87. self.log('OPERATIONPROFIT,GROSS%.2f,NET%.2f'%
88. (trade.pnl,trade.pnlcomm))
90. defnext(self):
91. #0.Checkifanorderispending...ifyes,wecannotsenda2ndone
92. #ifself.order:
93. #return
94. bar_time=self.datetime.datetime()
95. bar_time_str=bar_time.strftime('%Y-%m-%d')
96. trading_date=bar_time+datetime.timedelta(days=1)
97. #bar_time_str=(self.datetime.datetime()+datetime.timedelta(1)).strftime('%Y-%m-%d')
98. print(bar_time_str,self.value)
99. print("barday===:",bar_time_str,"===============")
100. for(k,v)inself.dataId_to_secId_dict.items():
101. size=self.positions[self.datas[k]].size
102. price=self.datas[k].close[-1]
103. self.positions_info.write('%s,%s,%s,%s,%s'%(bar_time_str,k,v,size,price))
104. self.positions_info.write('\n')
106. #1.nomattertheadjustmentday,up/downfloorblockedordershoulddealwitheachbar
107. for(k,v)inself.order_line_dict.items():
108. bar=self.datas[k]
109. buyable=Falseifbar.low[1]/bar.close[0]>1.0950elseTrue
110. sellable=Falseifbar.high[1]/bar.close[0]<0.910elseTrue
112. #buyable=Falseif(bar.open[1]/bar.close[0]>1.0950)and(bar.close[1]/bar.close[0]>1.0950)elseTrue
113. #sellable=Falseif(bar.open[1]/bar.close[0]<0.910and(bar.close[1]/bar.close[0]<0.91))elseTrue
115. ifv>0:
116. ifbuyable:
117. delself.order_line_dict[k]
118. self.log('%sBUYCREATE,%.2f,vlois%s'%(self.dataId_to_secId_dict[k],bar.open[1],v))
119. self.order=self.buy(data=bar,size=v,exectype=Order.Market)
120. else:
121. print("############:")
122. print("unbuyable:",self.dataId_to_secId_dict[k])
123. elifv<0:
124. ifsellable:
125. delself.order_line_dict[k]
126. self.log('%sSELLCREATE,%.2f,volis%s'%(self.dataId_to_secId_dict[k],bar.open[1],v))
127. self.order=self.sell(data=bar,size=abs(v),exectype=Order.Market)
128. else:
129. print("############:")
130. print("unsellable:",self.dataId_to_secId_dict[k])
132. #2.ensuretheadjustmentday
133. #2.1getthecurrentbartime
134. adj_sig=self.adj_df[self.adj_df['sigdate']==trading_date.strftime('%Y-%m-%d')][['secucode','hl_weight']]
135. #2.2checktheadjustmentday
136. iflen(adj_sig)==0orbar_time_str==self.end_date:
137. return
138. #3.adjusttheportfolio
139. #3.1settwodictstostorethebuyorderandsellorderspearately
140. buy_dict={}
141. sell_dict={}
142. self.order_line_dict={}
143. current_position_data_id=list()
144. #3.2iteratetheportfolioinstrumentsanddivideintobuygroupandsellgroup
145. forindexinadj_sig.index:
146. #getcurrentinstrumentcodeandtransfertothedata_id
147. sec_id=adj_sig.loc[index]['secucode']
148. data_id=self.secId_to_dataId_dict[sec_id]
149. ifself.backtesting_lengthanddata_id>=self.backtesting_length:
150. continue
151. bar=self.datas[data_id]
152. current_position_data_id.append(data_id)
153. #getthetargetweightvalue
154. target_weight=adj_sig.loc[index]['hl_weight']
155. #calculatethecurrentweightvalue
156. current_position=self.positions[self.datas[data_id]]
157. current_mv=current_position.size*bar.close[0]
158. current_weight=current_mv/float(self.value)
159. diff_weight=(target_weight-current_weight)
160. ifbar.open[1]==0:
161. continue
162. diff_volume=int(diff_weight*self.value/bar.open[1]/100)*100
163. print("theweightdifference",diff_weight)
164. ifdiff_volume>0:
165. buy_dict[data_id]=diff_volume
166. elifdiff_volume<0:
167. sell_dict[data_id]=diff_volume
169. #3.3makeorderwork
170. for(k,v)insell_dict.items():
171. bar=self.datas[k]
172. #sellable=Falseif(bar.high[1]==bar.low[1])and(bar.open[1]/bar.close[0]<0.920)elseTrue
173. sellable=Falseif(bar.open[1]/bar.close[0]<0.920)elseTrue
174. #sellable=Falseif(bar.open[1]/bar.close[0]<0.910and(bar.close[1]/bar.close[0]<0.91))elseTrue
176. ifsellable:
177. self.log('%sSELLCREATE,%.2f,volis%s'%(self.dataId_to_secId_dict[k],bar.open[1],v))
178. self.order=self.sell(data=bar,size=abs(v),exectype=Order.Market)
179. else:
180. print("############:")
181. print("unsellable:",self.dataId_to_secId_dict[k])
182. self.order_line_dict[k]=v
184. for(k,v)inbuy_dict.items():
185. bar=self.datas[k]
186. #buyable=Falseif(bar.low[1]==bar.high[1])and(bar.open[1]/bar.close[0])>1.0950elseTrue
187. buyable=Falseif(bar.open[1]/bar.close[0])>1.0950elseTrue
188. #buyable=Falseif(bar.open[1]/bar.close[0]>1.0950)and(bar.close[1]/bar.close[0]>1.0950)elseTrue
190. ifbuyable:
191. self.log('%sBUYCREATE,%.2f,vlois%s'%(self.dataId_to_secId_dict[k],bar.open[1],v))
192. self.order=self.buy(data=bar,size=v,exectype=Order.Market)
193. else:
194. print("############:")
195. print("unbuyable:",self.dataId_to_secId_dict[k])
196. self.order_line_dict[k]=v
198. #3.4closeposition,whenthedata_idisnotincurrentportfolioandinlastportfolio,weclosetheposition
199. ifself.pre_position_data_id:
200. clost_data_id=[difordiinself.pre_position_data_idifdinotincurrent_position_data_id]
201. fordiinclost_data_id:
202. bar=self.datas[di]
203. print('CLOSEPOSITION:',self.dataId_to_secId_dict[di])
204. self.order=self.close(data=bar)
205. #3.5updatethepositiondataidfornextchecking
206. self.pre_position_data_id=current_position_data_id
208. defstop(self):
209. #plotthenetvalueandthebenchmarkcurves
210. plot_df=pd.concat([pd.Series(self.value_for_plot,).to_frame(),self.benchmark],axis=1,join='inner')
211. plot_df.to_csv('result.csv')
212. self.positions_info.close()
213. print("death")
215. defbacktrader_backtest(start_date,end_date,trading_csv_name,portfolio_csv_name,bechmark_csv_name):
216. #1.backtestparameterssetting:starttime,endtime,theassetsnumber(backtest_length)andthebenckmarkdataandnewacerebro
217. start_date,end_date=datetime.datetime.strptime(start_date,"%Y-%m-%d"),datetime.datetime.strptime(end_date,"%Y-%m-%d")
218. backtest_length=None
219. #benchmarkseries
220. ifbechmark_csv_name:
221. benchmark=pd.read_csv(bechmark_csv_name,date_parser=True,dtype={'date':str})
222. benchmark['date']=benchmark['date'].apply(lambdax:datetime.datetime.strptime(x,"%Y-%m-%d"))
223. benchmark=benchmark.set_index('date')
224. else:
225. benchmark=None
227. #result_df=pd.DataFrame()
228. cerebro=bt.Cerebro()
230. #2.getrequiredtradingdata
231. #2.1gettradingdata(totaltradingdata)
232. trading_data_df=pd.read_csv(trading_csv_name,dtype={'sigdate':str,'secucode':str})
233. trading_data_df.rename(columns={'sigdate':'tradingdate','secucode':'ticker'},inplace=True)
234. transer=lambdax:datetime.datetime.strptime(x,"%Y-%m-%d")
235. trading_data_df['tradingdate']=trading_data_df['tradingdate'].apply(transer)
236. trading_data_df=trading_data_df[(trading_data_df['tradingdate']>start_date)&(trading_data_df['tradingdate']
237. trading_data_df['openinterest']=0
238. trading_data_df=trading_data_df.set_index('tradingdate')
239. #2.2gettargetportfoliodataforthetargetassetsfilter
240. adj_df=pd.read_csv(portfolio_csv_name,dtype={'secucode':str})
241. adj_df=adj_df[['sigdate','secucode','hl_weight']]
243. parser1=lambdax:datetime.datetime.strptime(x,"%Y/%m/%d")
244. parser2=lambdax:x.strftime("%Y-%m-%d")
245. adj_df['sigdate']=adj_df['sigdate'].apply(parser1)
246. adj_df=adj_df[(adj_df['sigdate']>start_date)&(adj_df['sigdate']
248. adj_df['sigdate']=adj_df['sigdate'].apply(parser2)
249. #2.3generatetwodictiontotransbetweensecIdandbacktraderid
250. sec_id_list=adj_df['secucode'].drop_duplicates().tolist()
251. data_id_list=[iforiinrange(len(sec_id_list))]
253. dataId_to_secId_dict=dict(zip(data_id_list,sec_id_list))
254. secId_to_dataId_dict=dict(zip(sec_id_list,data_id_list))
256. print("totalstocks'number",len(sec_id_list),'.','datafeeding......')
257. #2.4feedrequireddatafeedandaddthemtothecerebro
258. forindex,sec_idinenumerate(sec_id_list[0:backtest_length]):
259. sec_df=trading_data_df[trading_data_df['ticker']==sec_id]
260. #sec_df=sec_df.set_index('tradingdate')
261. sec_raw_start=sec_df.index[0]
262. ifsec_raw_start!=start_date.strftime("%Y-%m-%d"):
263. na_fill_value=sec_df.head(1)['open'].values[0]
264. df_temp=pd.DataFrame(index=pd.date_range(start=start_date,end=sec_raw_start,freq='D')[:-1],
265. columns=['open','high','low','close','volume','openinterest']
266. ).fillna(na_fill_value)
267. frames=[df_temp,sec_df]
268. sec_df=pd.concat(frames)
269. data_feed=bt.feeds.PandasData(dataname=sec_df,
270. fromdate=start_date,
271. todate=end_date
272. )
273. cerebro.adddata(data_feed,name=sec_id)
274. print('datafeedfinish!')
275. #3.cereberoconfig
276. cerebro.addstrategy(AlphaPortfolioStrategy,
277. dataId_to_secId_dict,secId_to_dataId_dict,adj_df,end_date.strftime("%Y-%m-%d"),backtest_length,benchmark)
278. cerebro.broker.setcash(20000000.0)
279. #cerebro.broker.setcash(10000000.0)
281. cerebro.broker.setcommission(commission=0.0008)
282. cerebro.broker.set_slippage_fixed(0.02)
283. cerebro.addanalyzer(bt.analyzers.Returns,_)
284. cerebro.addanalyzer(bt.analyzers.SharpeRatio,_name='SharpeRatio',riskfreerate=0.00,stddev_sample=True,annualize=True)
285. cerebro.addanalyzer(bt.analyzers.AnnualReturn,_name='AnnualReturn')
286. cerebro.addanalyzer(bt.analyzers.DrawDown,_name='DW')
287. cerebro.addanalyzer(bt.analyzers.TradeAnalyzer,_name='TradeAnalyzer')
288. ###informationprint
289. start_cash=(cerebro.broker.getvalue()-10000000.0)
290. #start_cash=(cerebro.broker.getvalue())
292. #print('StartingPortfolioValue:%.2f'%(cerebro.broker.getvalue()-10000000.0))
293. print("startcerebro.run()")
294. results=cerebro.run()#runthecerebro
295. #4.showtheresult
296. strat=results[0]
297. final_value=(cerebro.broker.getvalue()-10000000.0)
298. total_return=((cerebro.broker.getvalue()-10000000.0)/10000000.0-1)
299. #final_value=(cerebro.broker.getvalue())
300. #total_return=((cerebro.broker.getvalue())/10000000.0-1)
302. sharpe_ratio=strat.analyzers.SharpeRatio.get_analysis()['sharperatio']
303. max_drowdown=strat.analyzers.DW.get_analysis()['max']['drawdown']
304. max_drowdown_money=strat.analyzers.DW.get_analysis()['max']['moneydown']
305. trade_info=strat.analyzers.TradeAnalyzer.get_analysis()
306. return_dict={'start_cash':start_cash,'final_value':final_value,'total_return':total_return,\
307. 'sharpe_ratio':sharpe_ratio,'max_drowdown':max_drowdown,'max_drowdown_money':max_drowdown_money,\
308. 'trade_info':trade_info}
309. returnreturn_dict

基本上，代码的注释很全面了，基本实现了涨跌停不能买入，进入排队等待，当日开盘价买入，滑点设置等等这些功能。Backtrader有一点，可能是为了加快速度，特别不方便，就是datafeed不能按照名字来实现查找，而是用index来寻找，所以我们需要建立一个全局的dict，能够实现data id到股票代码以及反过来的一一对应的dict。别的，其实实现起来还是比较方便的，但是性能有待提高，等后续需要继续启动这个项目的时候，笔者继续努力吧。譬如如何修改一个查找的逻辑和数据类型的使用。