Migration from CVS to SVN

From GRASS-Wiki
Revision as of 23:00, 5 December 2007 by ⚠️Landa (talk | contribs) (→‎Testing SVN repository: moved to discussion page)
Jump to navigation Jump to search

This page contains notes related to GRASS code migration (planned) from CVS to SVN.

Basic

  • The SVN command line interface is just like CVS, many tasks are identical- just change the program name from cvs to svn.

Gotchas

  • cvs2svn is known to break binary files (images) which were not imported into the CVS with the -kb flag. Luckily Glynn fixed most of these some months ago.
  • Files using keyword substitution, such as $Date$ in the description.html files, will have to have support for that enabled manually, once per file (or write a find routine with | xargs svn ...).
$ svn propset svn:keywords "Date" filename.txt
$ svn commit

or 

$ find . -name '*.c' | xargs svn propset svn:keywords "Date" 
$ find . -name '*.html' | xargs svn propset svn:keywords "Date" 
$ svn commit
  • how to maintain timestamps of files? We want to keep the last modification date, not the date of local download

GRASS CVS repository structure

/grass-cvs
|
+---/CVSROOT
|
+---/grass
|
+---/grass51
|
+---/grass6
|
+---/grass_doc
|
+---/libgrass
|
+---/newsletter
|
+---/programgrass50
|
+---/web



GRASS OSGeo SVN repository structure

→ Scenario 2

Scripts: 

./grass-cvs2svn-base.sh
./grass-cvs2svn-s2.sh
./grass-cvs2svn-wxpython.sh

→ grass-svn2

GRASS 7 development notes

MIME types

According to Converting CVS to subversion: 

CVSREPOS="$(pwd)/grass-cvs"
PROJECTNAME=grass6

# Find all extensions. Also include filenames without extension.
# The E (extension) and S (slash) trick is to get GNU sort to separate
# them, although this is not really necessary. But note that at the
# same time it removes the leading slash from filenames without extension.
find $CVSREPOS/$PROJECTNAME -type f -name '*,v' ! -name '.cvsignore,v' | \
   sed -e 's%.*\([./][^.]*\),v$%\1%' -e 's/\./E/' -e 's/\//S/' | \
   sort -u | sed -e 's/^S//' -e 's/^E/./' > step1

# Compose an extended regular expression that matches any "extension"
# as found by the previous step.
EXT1="($(grep '^\.' step1 | xargs echo | sed -e 's/^\.//' -e 's/\+/\\+/g' -e 's/ \./|/g'))"

# Find all mime-types and related extensions that really exist.
egrep -i '^alnum:[^[:space:]]*space:+([^[:space:]]+ )*'"$EXT1"'($| )' /etc/mime.types > step2

# Extract the list of extensions from the previous step,
# filtering out the extensions that we don't have.
for ext in $(sed -re 's/^[^[:space:]]*space:+//' step2); do echo $ext; done | \
   egrep -i '^'"$EXT1"'$' | sort -u > step3

# Compose an extended regular expression from the previous step.
EXT2="($(cat step3 | xargs echo | sed -e 's/ /|/g'))"
 
# Find all "extensions" that weren't really extensions
# (or for which we don't know a MIME type).
grep '^\.' step1 | egrep -iv '^\.'"$EXT2"'$' > step4

# And turn it into an extended regular expression.
EXT3="($(sed -e 's/\./\\\\./' step4 | xargs echo | sed -e 's/ /|/g'))"

# Create a list of files for which no MIME type is known.
find $CVSREPOS/$PROJECTNAME -type f -name '*,v' ! -name '.cvsignore,v' | \
   sed -e 's%.*/\([^/]*\),v$%\1%' | egrep -i "$EXT3"'$' | sort -u > step5

Binary files in 'step5': 

./macosx/app/app.icns:                                       data
./db/drivers/dbf/dbf_catalog/datetime.dbf:                   DBase 3 data file (2 records)
./imagery/i.atcorr/test_suite/ETM4_400x400_atms_corr.raw:    data
./imagery/i.atcorr/test_suite/ETM4_400x400.raw:              data
./macosx/app/English.lproj/MainMenu.nib/keyedobjects.nib:    Apple binary property list
./lib/proj/nzgd2kgrid0005.gsb:                               data
./db/drivers/dbf/dbf_catalog/river.dbf:                      DBase 3 data file (5 records)
./raster/r.slope.aspect/r_sl_asp_northangle_diffs.tar.gz:    gzip compressed data, was "r.slope.aspect-diffs.tar",  from Unix, last modified: Tue Jul 21 20:15:15 1998
./lib/vector/diglib/test.ok:                                 data

Add to mine.types: 

application/dbase                          dbf
application/x-gtar                         gtar tgz taz tar.gz
application/octet-stream                   bin icns raw nib gsb ok

# Create a map from extension to MIME type. If a MIME type that starts
# with 'text' exist, use that - otherwise use application/octet-stream
# when there is more than one MIME type, or use the single known MIME type.
for f in $(cat step3); do \
   MIMETYPES=$(egrep -i 'space:'$f'( |$)' step2 | sed -e 's/space:.*//'); \
   echo $f: $MIMETYPES; done | \
   sed -e 's%:.* \(text/[^ ]*\).*%: \1%' -e 's%: [^ ].* .*%: application/octet-stream%' > step6

? b: chemical/x-molconn-Z (only ./raster/r.le/r.le.setup/polytocell/bmf.b, I guess it should be bmf.c...)
grass6/raster/r.le/r.le.setup/polytocell/bmf.b:               ASCII C program text
grass/src/raster/r.le/r.le.setup/polytocell/bmf.b:            ASCII C program text
grass/src/imagery/i.points3/inter/find.b:                     ASCII C program text
grass/src.contrib/SCS/paint/Programs/newp.map/cmd/scan_gis.b: ASCII C program text
grass/src.contrib/SCS/paint/Programs/newp.map/cmd/map.b:      ASCII C program text
grass/src.contrib/CERL/SGI/ISM/grid/gdwrit.b:                 ASCII text
grass/src.garden/grass.hdf/hdf3/HDF.lib.3.2.3/doc/HDF.apdx.b: ASCII English text
-> b: text/plain

? bak: application/x-trash
grass/src.contrib/GMSL/sg4d/lightdefs.bak:                            ASCII C program text
grass/src.contrib/GMSL/g3d/src3d/raster/r3.showdspf.openGL/Viz.h.bak: ASCII C program text
-> bak: text/plain

? bat: application/x-msdos-program
grass6/scripts/windows_launch.bat:                            ASCII text
grass6/lib/init/grass.bat:                                    MS-DOS batch file text
grass6/lib/init/init.bat:                                     MS-DOS batch file text
grass6/lib/init/grass-run.bat:                                MS-DOS batch file text
grass6/visualization/nviz/scripts/nviz.bat:                   ASCII text
grass/src.contrib/CERL/raster/r.rational.regression/main.bat: ASCII C program text
grass/cygwin/startxgrass.bat:                                 MS-DOS batch file text
grass/cygwin/startxwingrass.bat:                              MS-DOS batch file text
-> bat: text/plain

bin: application/octet-stream
grass/src/general/g.help/help/06.export/06.02.asc.bin: ASCII English text
grass/src/raster/r.out.bin:                            directory
grass/src/raster/r.in.bin:                             directory
grass/src/tcltkgrass/module/r.out.bin:                 ASCII text
grass/src/tcltkgrass/module/r.in.bin:                  ASCII English text
grass/locale/ru/tcltkgrass/module/r.out.bin:           ISO-8859 text
grass/locale/ru/tcltkgrass/module/r.in.bin:            ISO-8859 text
grass6/raster/r.out.bin:                               directory
grass6/raster/r.in.bin:                                directory
-> bin: text/plain

cab: application/x-cab
grass/src.contrib/eurogis/v.in.EE/x.cab: ASCII text
-> cab: text/plain

cat: application/vnd.ms-pki.seccat
grass/src/scripts/contrib/i.oif/i.oif.cat: ASCII English text
-> cat: text/plain

application/java-vm
grass/src/general/g.help/help/09.imagery/09.04.class:       ASCII English text
grass/src/general/g.help/help/17.manual/Help.pages/i.class: ASCII English text
grass/src/imagery/i.class:                                  directory
grass6/imagery/i.class:                                     directory
-> class: text/plain

c: text/x-csrc
cc: text/x-c++src
cpp: text/x-c++src
css: text/css
csv: text/csv

? dat: chemical/x-mopac-input
grass6/misc/m.cogo/cogo.dat:                           ASCII text
grass6/lib/proj/ntv1_can.dat:                          data
grass6/lib/gis/fmode.dat:                              ASCII C program text
grass6/raster/r.statistics/gauss.dat:                  ASCII text
grass/src/libes/proj/ntv1_can.dat:                     data
grass/src/sites/s.qcount/tutorial/cls.dat:             ASCII text
grass/src/sites/s.qcount/tutorial/reg.dat:             ASCII text
grass/src/sites/s.qcount/tutorial/csr.dat:             ASCII text
grass/src/misc/m.cogo/cogo.dat:                        ASCII text
grass/src/raster/r.in.gridatb/example/elev.dat:        ASCII text
grass/src/raster/r.statistics/cmd/gauss.dat:           ASCII text
grass/src/paint/Drivers/versatec/patterns/ce3200.dat:  ASCII English text
grass/src.contrib/PURDUE/s.medp/doc/cressie.dat:       ASCII English text
grass/src.contrib/CERL/raster/nodenumber/xsect.dat:    ASCII text
grass/src.contrib/CERL/raster/nodenumber/yak_trap.dat: ASCII text
-> plain/text / application/octet-stream

dbf: application/dbase

? dir: application/x-director
grass6/raster/r.fill.dir:                     setgid directory
grass6/tools/cvs.rename.dir:                  POSIX shell script text executable
grass/src/raster/r.fill.dir:                  setgid directory
grass/src/tcltkgrass/module/r.fill.dir:       ASCII text
-> dir: text/plain
 
? doc: application/msword
grass/src/fonts/original/hershey.doc:                        ASCII English text
grass/src.contrib/SDTS/libes/fips123/bugs.doc:               data
grass/src.contrib/SDTS/libes/fips123/fixes.doc:              data
grass/src.contrib/SDTS/mapdev/v.in.sdts/filenames.doc:       ASCII English text
grass/src.nonGPL/agnps/r.agnps50/documentation/vrfsftwr.doc: Microsoft Word 6.0 Document

eps: application/postscript
fig: application/x-xfig

frame: application/x-maker
grass/src/display/d.frame:                                  directory
grass/src/general/g.help/help/12.map.display/12.03.d.frame: ASCII English text
grass/src/general/g.help/help/17.manual/Help.pages/d.frame: ASCII English text
grass/src/tcltkgrass/module/d.frame:                        ASCII English text
grass/locale/ru/tcltkgrass/module/d.frame:                  ISO-8859 text
grass/unused/tcltkgrass/module/d.frame:                     ASCII English text
grass6/display/d.frame:                                     directory
-> text/plain

frm: application/x-maker
grass/src/general/g.help/help/04.wind.mgmt/04.02.mng.frm: ASCII English text
-> text/plain 

gen: chemical/x-genbank
grass/src/display/devices/windows/libW11/wrap/xwrappers.gen: ASCII C program text, with very long lines
-> text/x-csrc

hdf: application/x-hdf
grass/src/tcltkgrass/module/r.out.hdf:         ASCII text
grass/src/tcltkgrass/module/r.in.hdf:          ASCII text
grass/src.garden/grass.hdf:                    directory
grass/src.garden/grass.hdf/hdf3/old/r.out.hdf: directory
grass/src.garden/grass.hdf/hdf3/old/r.in.hdf:  directory
grass/src.garden/grass.hdf/hdf3/r.out.hdf:     directory
grass/src.garden/grass.hdf/hdf3/r.in.hdf:      directory
grass/src.garden/grass.hdf/hdf4/r.out.hdf:     directory
grass/src.garden/grass.hdf/hdf4/r.in.hdf:      directory
grass/src.garden/grass.hdf/hdf5/r.out.hdf:     directory
grass/src.garden/grass.hdf/hdf5/r.in.hdf:      directory
grass/locale/ru/tcltkgrass/module/r.out.hdf:   ISO-8859 text
grass/locale/ru/tcltkgrass/module/r.in.hdf:    ISO-8859 text
-> hdf: text/plain

gif: image/gif
gsb: application/octet-stream
h: text/x-chdr
hh: text/x-c++hdr
htm: text/html
html: text/html
icns: application/octet-stream
ico: image/x-icon
jpg: image/jpeg
lyx: application/x-lyx
man: application/x-troff-man

me: application/x-troff-me
grass/src.contrib/SCS/paint/Programs/ps.map/read.me: ASCII English text
-> text/plain

mid: audio/midi
grass/src/CMD/generic/make.mid: ASCII English text
-> text/plain

mpeg: video/mpeg
grass/src/raster/r.out.mpeg:            directory
grass/src/tcltkgrass/module/r.out.mpeg: ASCII English text
grass6/raster/r.out.mpeg:               directory
-> text/plain

application/x-troff-ms
grass/unused/man/utilities/ptx.to.ms: ASCII text
-> text/plain

nib: application/octet-stream
ok: application/octet-stream

? old: application/x-trash
grass6/dist.i686-pc-linux-gnu/docs/html/gem/img1.old: PNG image data, 559 x 111, 8-bit colormap, interlaced
grass6/gem/docs/GEM-Manual/img1.old:                  PNG image data, 559 x 111, 8-bit colormap, interlaced
grass/unused/misc/m.clump/proto.h.old:                ASCII C program text
grass/src/raster/r.random.surface/MAN.old:            troff or preprocessor input text
grass/src/tcltkgrass/README.old:                      ASCII English text
-> old: image/png for grass6 old: text/plain for grass5

pat: image/x-coreldrawpattern
grass/src/paint/Drivers/versatec/patterns/ce3200.pat: ASCII text
-> text/plain

patch: text/x-diff
pdf: application/pdf
pl: text/x-perl
pm: text/x-perl

pbm: image/x-portable-bitmap
grass/src/raster/r.in.pbm:            directory
grass/src/tcltkgrass/module/r.in.pbm: ASCII text
-> text/plain

pgm: image/x-portable-grayma
grass/src/raster/r.in.pgm:            directory
grass/src/tcltkgrass/module/r.in.pgm: ASCII text
-> text/plain

png: image/png

ppm: image/x-portable-pixmap
grass/src/paint/Programs/p.ppm:                           directory
grass/src/general/g.help/help/17.manual/Help.pages/p.ppm: ASCII text
grass/src/raster/r.out.ppm:                               directory
grass/src/raster/r.in.ppm:                                directory
grass/src/raster/r.in.ppm/trees24bit.ppm:                 Netpbm PPM "rawbits" image data
grass/src/raster/r.in.ppm/image24bit.ppm:                 Netpbm PPM "rawbits" image data
grass/src/tcltkgrass/module/r.out.ppm:                    ASCII text
grass/src/tcltkgrass/module/r.in.ppm:                     ASCII English text
grass/src.contrib/SCS/paint/Drivers/grey.ppm:             directory
grass/locale/ru/tcltkgrass/module/r.out.ppm:              ISO-8859 text
grass/locale/ru/tcltkgrass/module/r.in.ppm:               ISO-8859 text
-> text/plain / image/x-portable-pixmap

ps: application/postscript
py: text/x-python

raw: application/octet-stream
grass/src/mapdev/v.in.arc/testdata/f1.txt.raw:                 ASCII text
grass/src/mapdev/v.in.arc/testdata/f2.txt.raw:                 ASCII text
grass6/imagery/i.atcorr/test_suite/ETM4_400x400.raw:           data
grass6/imagery/i.atcorr/test_suite/ETM4_400x400_atms_corr.raw: data
-> text/plain for grass5

rgb: image/x-rgb
grass/src/display/d.rgb:                                      directory
grass/src/libes/g3d/snap.rgb:                                 SGI image data, RLE, 3-D, 573 x 372, 3 channels
grass/src/general/g.help/help/17.manual/Help.pages/d.rgb:     ASCII English text
grass/src/general/g.help/help/17.manual/Help.pages/i.his.rgb: ASCII text
grass/src/general/g.help/help/17.manual/Help.pages/r.his.rgb: ASCII text
grass/src/tcltkgrass/module/d.rgb:                            ASCII text
grass/src/tcltkgrass/module/i.his.rgb:                        ASCII text
grass/src.contrib/GMSL/NVIZ2.2/TOGL/apps/ben.rgb:             SGI image data, RLE, 3-D, 133 x 158, 3 channels
grass/locale/ru/tcltkgrass/module/d.rgb:                      ISO-8859 text
grass/locale/ru/tcltkgrass/module/i.his.rgb:                  ISO-8859 text
grass/unused/tcltkgrass/module/d.rgb:                         ASCII English text
grass/unused/tcltkgrass/module/i.his.rgb:                     ASCII English text
grass6/lib/g3d/snap.rgb:                                      SGI image data, RLE, 3-D, 573 x 372, 3 channels
grass6/display/d.rgb:                                         directory
grass6/scripts/i.landsat.rgb:                                 directory
grass6/scripts/i.landsat.rgb/i.landsat.rgb:                   Bourne shell script text executable
grass6/imagery/i.his.rgb:                                     directory
-> text/plain / image/x-rgb

ros: chemical/x-rosdal
grass/src/raster/wildfire/src/r.ros: directory
grass/src/tcltkgrass/module/r.ros:   ASCII English text
grass6/raster/wildfire/r.ros:        directory
-> text/plain

rtf: application/rtf
sh: text/x-sh

? src: application/x-wais-source
grass6/lib/init/grass-run.src:        POSIX shell script text executable
grass6/lib/init/grass.src:            POSIX shell script text executable
grass6/binaryInstall.src:             POSIX shell script text executable
grass/src/general/init/grass.src:     POSIX shell script text executable
grass/src/tcltkgrass/gis_set.tcl.src: ASCII English text
grass/binaryInstall.src:              POSIX shell script text executable
-> src: text/plain

? t: application/x-troff
grass6/swig/perl/t/R_slope_aspect.t: ASCII English text
-> t: text/plain

tcl: text/x-tcl
tex: text/x-tex
tgz: application/x-gtar
tiff: image/tiff
txt: text/plain
xbm: image/x-xbitmap

? xyz: chemical/x-xyz
grass6/scripts/r.out.xyz:                        setgid directory
grass6/scripts/r.out.xyz/r.out.xyz:              POSIX shell script text executable
grass6/raster/r.in.xyz:                          setgid directory
grass/src/raster/r.out.xyz:                      setgid directory
grass/src/tcltkgrass/module/r.out.xyz:           ASCII text
-> xyz: text/plain

# Instead, build a props file for use with the '--auto-props=FILE' option.
echo "[auto-props]" > propsfile-g5
sed -e 's%: \(text/.*\)% = svn:eol-style=native;svn:keywords=Author Date Id Revision;svn:mime-type=\1%' \
-e 's/^/*./' -e 's/: / = svn:mime-type=/' step6-g5 >> propsfile-g5

echo "[auto-props]" > propsfile-g6
sed -e 's%: \(text/.*\)% = svn:eol-style=native;svn:keywords=Author Date Id Revision;svn:mime-type=\1%' \
-e 's/^/*./' -e 's/: / = svn:mime-type=/' step6-g6 >> propsfile-g6

Remove mime-type:text/* 

cat propsfile-g6 | sed -e 's/svn:mime-type=text\/.*//g' > propsfile-g6-1
cat propsfile-g5 | sed -e 's/svn:mime-type=text\/.*//g' > propsfile-g5-1

#Finish the generation of propsfile
sed -re 's/(.*\.([^.]*$))/\2 \1/' -e 's/^([^.]*)$/\1 \1/' step5-g6 | sort | \
sed -e 's/^[^ ]* //' -e 's%$% = svn:eol-style=native;svn:keywords=Author Date Id Revision%' >> propsfile-g6-1

#Finish the generation of propsfile
sed -re 's/(.*\.([^.]*$))/\2 \1/' -e 's/^([^.]*)$/\1 \1/' step5-g5 | sort | \
sed -e 's/^[^ ]* //' -e 's%$% = svn:eol-style=native;svn:keywords=Author Date Id Revision%' >> propsfile-g5-1

Files are available here.

External links

SVN hosting

There are two main options to host the new SVN repository.

Retrieved from "https://grasswiki.osgeo.org/w/index.php?title=Migration_from_CVS_to_SVN&oldid=5475"