利用caffe的python接口进行训练时遇到的问题
来源：互联网 发布：淘宝新手卖家群 编辑：程序博客网 时间：2024/05/23 01:18
利用caffe的python接口来训练：python ./train.py, 想输出梯度，便在solver.cpp里做了修改，添加代码：
int max_len = 0; for (int l = 0; l < net_->layers().size(); ++l) { Layer<Dtype>& layer = *net_->layers()[l].get(); if (layer.layer_param().name().length() > max_len) max_len = layer.layer_param().name().length(); } // bottom blobs and their gradient const vector<vector<Blob<Dtype>*> >& bottom_vecs = net_->bottom_vecs(); for (int l = 0; l < net_->layers().size(); ++l) { Layer<Dtype>& layer = *net_->layers()[l].get(); for (int b=0; b<bottom_vecs[l].size(); ++b) { const Blob<Dtype>* blob = bottom_vecs[l][b]; Dtype data_sum = 0; Dtype diff_sum = 0; data_sum = blob->asum_data(); diff_sum = blob->asum_diff(); data_sum /= blob->count(); diff_sum /= blob->count(); LOG_IF(INFO, Caffe::root_solver()) << std::left << std::setw(max_len + 1) << std::setfill(' ') << layer.layer_param().name() << " bottom_blob " << b << ": " << std::scientific << data_sum << " [" << diff_sum << "]"; } } LOG(INFO) << std::endl; // output learnable weights and their gradients for (int l = 0; l < net_->layers().size(); ++l) { Layer<Dtype>& layer = *net_->layers()[l].get(); for (int bp = 0; bp < layer.blobs().size(); ++bp) { Blob<Dtype>& para_blob = *layer.blobs()[bp].get(); const Dtype* para_blob_cpu_data = para_blob.cpu_data(); const Dtype* para_blob_cpu_diff = para_blob.cpu_diff(); Dtype para_data_sum = 0; Dtype para_diff_sum = 0; for (int i = 0; i < para_blob.count(); ++i) { para_data_sum += (para_blob_cpu_data[i] > Dtype(0.)) ? para_blob_cpu_data[i] : - para_blob_cpu_data[i]; para_diff_sum += (para_blob_cpu_diff[i] > Dtype(0.)) ? para_blob_cpu_diff[i] : - para_blob_cpu_diff[i]; } para_data_sum /= para_blob.count(); para_diff_sum /= para_blob.count(); LOG_IF(INFO, Caffe::root_solver()) << std::left << std::setw(max_len + 1) << std::setfill(' ') << layer.layer_param().name() << " param_blob " << bp << ": " << std::scientific << para_data_sum << " [" << para_diff_sum << "]"; } } // end add by Zzp然而，继续运行train.py，并没有输出梯度，找了很久的原因才发现，原来_caffe.so已经被安装到系统路径中，即‘/usr/lib/python2.7/dist-packages/caffe’中，所以在运行train.py的时候无论你对caffe的源码做了什么样的修改，都会默认调用系统路径里的库。所以，需要将/usr/lib/python2.7/dist-packages 中的caffe库全部删掉，然后用PYTHONPATH指定路径。
1 进入/usr/lib/python2.7/dist-packages ， ls
mini@mini:/usr/lib/python2.7/dist-packages$ lsadium_theme_ubuntu-0.3.4.egg-info PAM.x86_64-linux-gnu.soANSI.py pandasANSI.pyc pandas-0.13.1.egg-infoantlr.py patsyantlr.pyc patsy-0.2.1.egg-infoapt pcardext.laaptdaemon pcardext.soapt_inst.so pep8-1.4.6.egg-infoapt_pkg.so pep8.pyaptsources pep8.pycapt_xapian_index-0.45.egg-info pexpectastroid pexpect-3.1.egg-infoastroid-1.0.1.egg-info PILaxi PILcompatbeautifulsoup4-4.2.1.egg-info PILcompat.pthblinker Pillow-2.3.0.egg-infoblinker-1.3.egg-info piston_mini_clientboost piston_mini_client-0.7.5.egg-infobs4 pkg_resources.pycaffe pkg_resources.pyccairo protobuf-2.5.0.egg-infochardet psutilchardet-2.0.1.egg-info psutil-1.2.1.egg-infoCommandNotFound _psutil_linux.x86_64-linux-gnu.socommand_not_found-0.3.egg-info _psutil_posix.x86_64-linux-gnu.soCrypto pxssh.pycupsext.la pxssh.pyccupsext.so pycrypto-2.6.1.egg-infocupshelpers pycups-1.9.66.egg-infocups.so pydot-1.0.28.egg-infocvxopt pydot.pycvxopt-1.1.4.egg-info pydot.pycCython pyflakesCython-0.20.1post0.egg-info pyflakes-0.8.1.egg-infocython.py pygmentscython.pyc Pygments-1.6.egg-infodateutil pygobject-3.12.0.egg-infodbus pygtkcompat_dbus_bindings.so pygtk.pth_dbus_glib_bindings.so pygtk.pydeb822.py pygtk.pycdeb822.pyc pyinotify-0.9.4.egg-infodebconf.py pyinotify.pydebconf.pyc pyinotify.pycdebian pylintdebian_bundle pylint-1.1.0.egg-infodebtagshw pyOpenSSL-0.13.egg-infodebtagshw-0.1.egg-info pyparsing-2.0.1.egg-infodecorator-3.4.0.egg-info pyparsing.pydecorator.py pyparsing.pycdecorator.pyc PyQt4defer pyserial-2.6.egg-infodefer-1.0.6.egg-info pysmbc-1.0.14.1.egg-infodirspec pysvndirspec-13.10.egg-info python_apt-0.9.3.5ubuntu2.egg-infodocutils python_dateutil-1.5.egg-infodocutils-0.11.egg-info python_debian-0.1.21_nmu2ubuntu2.egg-infodot_parser.py python_gflags-1.5.1.egg-infodot_parser.pyc python-support.pthdrv_libxml2.py pytzdrv_libxml2.pyc pytz-2012c.egg-infoduplicity pyxdg-0.25.egg-infoduplicity-0.6.23.egg-info pyximporteasy_install.py PyYAML-3.10.egg-infoeasy_install.pyc pyzmq-14.0.1.egg-infofdpexpect.py READMEfdpexpect.pyc reportlabflask reportlab-3.0.egg-infoFlask-0.10.1.egg-info requestsflaskext requests-2.2.1.egg-infoflask_socketio roman-2.0.0.egg-infoFlask_SocketIO-0.6.0.egg-info roman.pyflask_wtf roman.pycFlask_WTF-0.11.egg-info ropeFSM.py rope-0.9.2.egg-infoFSM.pyc sambagevent scanext.lagevent-1.0.egg-info scanext.sogevent_socketio-0.3.6.egg-info scikit_image-0.9.3.egg-infogeventwebsocket scikit_learn-0.14.1.egg-infogevent_websocket-0.9.3.egg-info scipygflags.py scipy-0.13.3.egg-infogflags.pyc screen.pygflags_validators.py screen.pycgflags_validators.pyc serialgi sessioninstallerglib sessioninstaller-0.0.0.egg-infogobject setuptoolsgoogle setuptools-3.3.egg-infogreenlet-0.4.2.egg-info setuptools.pthgreenlet.so simplegeneric-0.8.1.egg-infogtk-2.0 simplegeneric.pygtk-2.0-pysupport-compat.pth simplegeneric.pycgunicorn simplejsongunicorn-17.5.egg-info simplejson-3.3.1.egg-infogyp sipconfig_nd.pygyp-0.1.egg-info sipconfig_nd.pych5py sipconfig.pyh5py-2.2.1.egg-info sipconfig.pychgext sip.sohpmudext.la six-1.5.2.egg-infohpmudext.so six.pyhtml5lib six.pychtml5lib-0.999.egg-info skimagehttplib2 sklearnhttplib2-0.8.egg-info smbcibus _smbc.soindicator_keyboard socketioIPython softwarecenter_aptd_pluginsipython-1.2.1.egg-info software_center_aptd_plugins-0.0.0.egg-infoitsdangerous-0.22.egg-info sphinxitsdangerous.py Sphinx-1.2.2.egg-infoitsdangerous.pyc spyder-2.2.5.egg-infojinja2 spyderlibJinja2-2.7.2.egg-info spyderpluginsldb.so ssh_import_id-3.21.egg-info_ldb_text.py statsmodels_ldb_text.pyc statsmodels-0.5.0.egg-infoleveldb-0.1.egg-info system_service-0.1.6.egg-infoleveldb.so tableslibxml2mod.so tables-3.1.1.egg-infolibxml2.py talloc.solibxml2.pyc tdb.solmdb _tdb_text.pylmdb-0.87.egg-info _tdb_text.pyclockfile-0.8.egg-info twistedlockfile.py Twisted_Core-13.2.0.egg-infolockfile.pyc Twisted_Web-13.2.0.egg-infologilab ubuntu-sso-clientlogilab_common-0.61.0.egg-info ubuntu-sso-client.pthlsb_release.py UbuntuSystemServicelsb_release.pyc unity_lens_photos-1.0.egg-infolxml urllib3lxml-3.3.3.egg-info urllib3-1.7.1.egg-info_markerlib werkzeugmarkupsafe Werkzeug-0.9.4.egg-infoMarkupSafe-0.18.egg-info wtformsmercurial WTForms-2.0.1.egg-infomercurial-2.8.2.egg-info xapianmx xdgnetworkx xdiagnosenetworkx-1.8.1.egg-info xdiagnose-3.6.3build2.egg-infonose xlrdnose-1.3.1.egg-info xlrd-0.9.2.egg-infonumexpr xlwtnumexpr-2.2.2.egg-info xlwt-0.7.5.egg-infonumpy yamlnumpy-1.8.2.egg-info _yaml.sooauthlib zeitgeistoauthlib-0.6.1.egg-info zmqoneconf zopeoneconf-0.3.7.14.04.1.egg-info zope.interface-4.0.5.egg-infoOpenSSL zope.interface-4.0.5-nspkg.pth包含caffe。
2 进入/usr/lib/python2.7/dist-packages/caffe， lld _caffe.so
mini@min:/usr/lib/python2.7/dist-packages/caffe$ ldd _caffe.so | grep caffe libcaffe-nv.so.0.14 => /usr/lib/x86_64-linux-gnu/libcaffe-nv.so.0.14 (0x00007f655b84b000)#libcaffe被安装到了系统路径，所以在运行train.py的时候无论你对caffe的源码做了什么样的修改，都会默认调用系统路径里的库3 删除
mini@mini:/usr/lib/python2.7/dist-packages$ sudo rm -rf caffe 4 编辑 .bashrc，添加PYTHONPATH环境变量export PYTHONPATH='/home/mm/SoftWare/caffe/python'，
然后..bashrc,生效
然后popd
0 0
- 利用caffe的python接口进行训练时遇到的问题
- 使用caffe的Python接口进行训练测试
- Caffe 训练遇到的问题
- 配置caffe python接口时遇到的一些问题
- 使用caffe训练faster-rcnn时遇到的问题总结
- 使用caffe训练faster-rcnn时遇到的问题总结
- 使用caffe训练faster-rcnn时遇到的问题总结
- caffe的python接口学习（9）：Caffe在Python中使用内存数据(MemoryData)进行训练
- 使用caffe对mnist进行训练遇到的点点滴滴
- Caffe的Python接口进行Cifar10可视化
- 利用Anaconda配置python接口遇到的问题
- 利用caffe训练好的模型进行测试
- caffe利用训练好的模型进行实际测试
- caffe利用训练好的模型进行实际测试
- 【Caffe的C++接口使用说明(三)】Ubuntu14.04下Caffe利用训练好的模型进行分类的C++接口使用说明(三)
- 利用caffe日志进行测试精度训练损失等的画图（caffe训练结果可视化）
- caffe的python接口学习（3）：训练模型（training)
- caffe的python接口学习（3）：训练模型（training)
- app-config
- 端到端的OCR：基于CNN的实现
- 基于ant的jmeter自动化性能测试
- 响应者链（Responder Chain）
- 浅谈IQKeyboardManager第三方库的使用
- 利用caffe的python接口进行训练时遇到的问题
- org.springframework.web.multipart.MultipartException: The current request is not a multipart request
- Hibernate关联映射之多对多
- ViewPager+Fragment实现无限加载
- android源码编译出错，一键解决
- link.exe 的参数
- 字符串
- Maven核心概念
- 用到了base64转图片文件的函数，记录一下
