Migration from CVS to SVN: Difference between revisions

From GRASS-Wiki
Jump to navigation Jump to search
(→‎Testing SVN repository: moved to discussion page)
(→‎MIME types: moved to discussion page)
Line 60: Line 60:
[[Image:Grass7-development-svn.png|center]]
[[Image:Grass7-development-svn.png|center]]


== MIME types ==


According to [http://www.xs4all.nl/~carlo17/svn/cvs2svn.html Converting CVS to subversion]:
CVSREPOS="$(pwd)/grass-cvs"
PROJECTNAME=grass6
# Find all extensions. Also include filenames without extension.
# The E (extension) and S (slash) trick is to get GNU sort to separate
# them, although this is not really necessary. But note that at the
# same time it removes the leading slash from filenames without extension.
find $CVSREPOS/$PROJECTNAME -type f -name '*,v' ! -name '.cvsignore,v' | \
    sed -e 's%.*\([./][^.]*\),v$%\1%' -e 's/\./E/' -e 's/\//S/' | \
    sort -u | sed -e 's/^S//' -e 's/^E/./' > step1
# Compose an extended regular expression that matches any "extension"
# as found by the previous step.
EXT1="($(grep '^\.' step1 | xargs echo | sed -e 's/^\.//' -e 's/\+/\\+/g' -e 's/ \./|/g'))"
# Find all mime-types and related extensions that really exist.
egrep -i '^[[:alnum:]][^[:space:]]*[[:space:]]+([^[:space:]]+ )*'"$EXT1"'($| )' /etc/mime.types > step2
# Extract the list of extensions from the previous step,
# filtering out the extensions that we don't have.
for ext in $(sed -re 's/^[^[:space:]]*[[:space:]]+//' step2); do echo $ext; done | \
    egrep -i '^'"$EXT1"'$' | sort -u > step3
# Compose an extended regular expression from the previous step.
EXT2="($(cat step3 | xargs echo | sed -e 's/ /|/g'))"
 
# Find all "extensions" that weren't really extensions
# (or for which we don't know a MIME type).
grep '^\.' step1 | egrep -iv '^\.'"$EXT2"'$' > step4
# And turn it into an extended regular expression.
EXT3="($(sed -e 's/\./\\\\./' step4 | xargs echo | sed -e 's/ /|/g'))"
# Create a list of files for which no MIME type is known.
find $CVSREPOS/$PROJECTNAME -type f -name '*,v' ! -name '.cvsignore,v' | \
    sed -e 's%.*/\([^/]*\),v$%\1%' | egrep -i "$EXT3"'$' | sort -u > step5
Binary files in 'step5':
./macosx/app/app.icns:                                      data
./db/drivers/dbf/dbf_catalog/datetime.dbf:                  DBase 3 data file (2 records)
./imagery/i.atcorr/test_suite/ETM4_400x400_atms_corr.raw:    data
./imagery/i.atcorr/test_suite/ETM4_400x400.raw:              data
./macosx/app/English.lproj/MainMenu.nib/keyedobjects.nib:    Apple binary property list
./lib/proj/nzgd2kgrid0005.gsb:                              data
./db/drivers/dbf/dbf_catalog/river.dbf:                      DBase 3 data file (5 records)
./raster/r.slope.aspect/r_sl_asp_northangle_diffs.tar.gz:    gzip compressed data, was "r.slope.aspect-diffs.tar",  from Unix, last modified: Tue Jul 21 20:15:15 1998
./lib/vector/diglib/test.ok:                                data
Add to mine.types:
application/dbase                          dbf
application/x-gtar                        gtar tgz taz tar.gz
application/octet-stream                  bin icns raw nib gsb ok
# Create a map from extension to MIME type. If a MIME type that starts
# with 'text' exist, use that - otherwise use application/octet-stream
# when there is more than one MIME type, or use the single known MIME type.
for f in $(cat step3); do \
    MIMETYPES=$(egrep -i '[[:space:]]'$f'( |$)' step2 | sed -e 's/[[:space:]].*//'); \
    echo $f: $MIMETYPES; done | \
    sed -e 's%:.* \(text/[^ ]*\).*%: \1%' -e 's%: [^ ].* .*%: application/octet-stream%' > step6
? b: chemical/x-molconn-Z (only ./raster/r.le/r.le.setup/polytocell/bmf.b, I guess it should be bmf.c...)
grass6/raster/r.le/r.le.setup/polytocell/bmf.b:              ASCII C program text
grass/src/raster/r.le/r.le.setup/polytocell/bmf.b:            ASCII C program text
grass/src/imagery/i.points3/inter/find.b:                    ASCII C program text
grass/src.contrib/SCS/paint/Programs/newp.map/cmd/scan_gis.b: ASCII C program text
grass/src.contrib/SCS/paint/Programs/newp.map/cmd/map.b:      ASCII C program text
grass/src.contrib/CERL/SGI/ISM/grid/gdwrit.b:                ASCII text
grass/src.garden/grass.hdf/hdf3/HDF.lib.3.2.3/doc/HDF.apdx.b: ASCII English text
-> b: text/plain
? bak: application/x-trash
grass/src.contrib/GMSL/sg4d/lightdefs.bak:                            ASCII C program text
grass/src.contrib/GMSL/g3d/src3d/raster/r3.showdspf.openGL/Viz.h.bak: ASCII C program text
-> bak: text/plain
? bat: application/x-msdos-program
grass6/scripts/windows_launch.bat:                            ASCII text
grass6/lib/init/grass.bat:                                    MS-DOS batch file text
grass6/lib/init/init.bat:                                    MS-DOS batch file text
grass6/lib/init/grass-run.bat:                                MS-DOS batch file text
grass6/visualization/nviz/scripts/nviz.bat:                  ASCII text
grass/src.contrib/CERL/raster/r.rational.regression/main.bat: ASCII C program text
grass/cygwin/startxgrass.bat:                                MS-DOS batch file text
grass/cygwin/startxwingrass.bat:                              MS-DOS batch file text
-> bat: text/plain
bin: application/octet-stream
grass/src/general/g.help/help/06.export/06.02.asc.bin: ASCII English text
grass/src/raster/r.out.bin:                            directory
grass/src/raster/r.in.bin:                            directory
grass/src/tcltkgrass/module/r.out.bin:                ASCII text
grass/src/tcltkgrass/module/r.in.bin:                  ASCII English text
grass/locale/ru/tcltkgrass/module/r.out.bin:          ISO-8859 text
grass/locale/ru/tcltkgrass/module/r.in.bin:            ISO-8859 text
grass6/raster/r.out.bin:                              directory
grass6/raster/r.in.bin:                                directory
-> bin: text/plain
cab: application/x-cab
grass/src.contrib/eurogis/v.in.EE/x.cab: ASCII text
-> cab: text/plain
cat: application/vnd.ms-pki.seccat
grass/src/scripts/contrib/i.oif/i.oif.cat: ASCII English text
-> cat: text/plain
application/java-vm
grass/src/general/g.help/help/09.imagery/09.04.class:      ASCII English text
grass/src/general/g.help/help/17.manual/Help.pages/i.class: ASCII English text
grass/src/imagery/i.class:                                  directory
grass6/imagery/i.class:                                    directory
-> class: text/plain
c: text/x-csrc
cc: text/x-c++src
cpp: text/x-c++src
css: text/css
csv: text/csv
? dat: chemical/x-mopac-input
grass6/misc/m.cogo/cogo.dat:                          ASCII text
grass6/lib/proj/ntv1_can.dat:                          data
grass6/lib/gis/fmode.dat:                              ASCII C program text
grass6/raster/r.statistics/gauss.dat:                  ASCII text
grass/src/libes/proj/ntv1_can.dat:                    data
grass/src/sites/s.qcount/tutorial/cls.dat:            ASCII text
grass/src/sites/s.qcount/tutorial/reg.dat:            ASCII text
grass/src/sites/s.qcount/tutorial/csr.dat:            ASCII text
grass/src/misc/m.cogo/cogo.dat:                        ASCII text
grass/src/raster/r.in.gridatb/example/elev.dat:        ASCII text
grass/src/raster/r.statistics/cmd/gauss.dat:          ASCII text
grass/src/paint/Drivers/versatec/patterns/ce3200.dat:  ASCII English text
grass/src.contrib/PURDUE/s.medp/doc/cressie.dat:      ASCII English text
grass/src.contrib/CERL/raster/nodenumber/xsect.dat:    ASCII text
grass/src.contrib/CERL/raster/nodenumber/yak_trap.dat: ASCII text
-> plain/text / application/octet-stream
dbf: application/dbase
? dir: application/x-director
grass6/raster/r.fill.dir:                    setgid directory
grass6/tools/cvs.rename.dir:                  POSIX shell script text executable
grass/src/raster/r.fill.dir:                  setgid directory
grass/src/tcltkgrass/module/r.fill.dir:      ASCII text
-> dir: text/plain
 
? doc: application/msword
grass/src/fonts/original/hershey.doc:                        ASCII English text
grass/src.contrib/SDTS/libes/fips123/bugs.doc:              data
grass/src.contrib/SDTS/libes/fips123/fixes.doc:              data
grass/src.contrib/SDTS/mapdev/v.in.sdts/filenames.doc:      ASCII English text
grass/src.nonGPL/agnps/r.agnps50/documentation/vrfsftwr.doc: Microsoft Word 6.0 Document
eps: application/postscript
fig: application/x-xfig
frame: application/x-maker
grass/src/display/d.frame:                                  directory
grass/src/general/g.help/help/12.map.display/12.03.d.frame: ASCII English text
grass/src/general/g.help/help/17.manual/Help.pages/d.frame: ASCII English text
grass/src/tcltkgrass/module/d.frame:                        ASCII English text
grass/locale/ru/tcltkgrass/module/d.frame:                  ISO-8859 text
grass/unused/tcltkgrass/module/d.frame:                    ASCII English text
grass6/display/d.frame:                                    directory
-> text/plain
frm: application/x-maker
grass/src/general/g.help/help/04.wind.mgmt/04.02.mng.frm: ASCII English text
-> text/plain
gen: chemical/x-genbank
grass/src/display/devices/windows/libW11/wrap/xwrappers.gen: ASCII C program text, with very long lines
-> text/x-csrc
hdf: application/x-hdf
grass/src/tcltkgrass/module/r.out.hdf:        ASCII text
grass/src/tcltkgrass/module/r.in.hdf:          ASCII text
grass/src.garden/grass.hdf:                    directory
grass/src.garden/grass.hdf/hdf3/old/r.out.hdf: directory
grass/src.garden/grass.hdf/hdf3/old/r.in.hdf:  directory
grass/src.garden/grass.hdf/hdf3/r.out.hdf:    directory
grass/src.garden/grass.hdf/hdf3/r.in.hdf:      directory
grass/src.garden/grass.hdf/hdf4/r.out.hdf:    directory
grass/src.garden/grass.hdf/hdf4/r.in.hdf:      directory
grass/src.garden/grass.hdf/hdf5/r.out.hdf:    directory
grass/src.garden/grass.hdf/hdf5/r.in.hdf:      directory
grass/locale/ru/tcltkgrass/module/r.out.hdf:  ISO-8859 text
grass/locale/ru/tcltkgrass/module/r.in.hdf:    ISO-8859 text
-> hdf: text/plain
gif: image/gif
gsb: application/octet-stream
h: text/x-chdr
hh: text/x-c++hdr
htm: text/html
html: text/html
icns: application/octet-stream
ico: image/x-icon
jpg: image/jpeg
lyx: application/x-lyx
man: application/x-troff-man
me: application/x-troff-me
grass/src.contrib/SCS/paint/Programs/ps.map/read.me: ASCII English text
-> text/plain
mid: audio/midi
grass/src/CMD/generic/make.mid: ASCII English text
-> text/plain
mpeg: video/mpeg
grass/src/raster/r.out.mpeg:            directory
grass/src/tcltkgrass/module/r.out.mpeg: ASCII English text
grass6/raster/r.out.mpeg:              directory
-> text/plain
application/x-troff-ms
grass/unused/man/utilities/ptx.to.ms: ASCII text
-> text/plain
nib: application/octet-stream
ok: application/octet-stream
? old: application/x-trash
grass6/dist.i686-pc-linux-gnu/docs/html/gem/img1.old: PNG image data, 559 x 111, 8-bit colormap, interlaced
grass6/gem/docs/GEM-Manual/img1.old:                  PNG image data, 559 x 111, 8-bit colormap, interlaced
grass/unused/misc/m.clump/proto.h.old:                ASCII C program text
grass/src/raster/r.random.surface/MAN.old:            troff or preprocessor input text
grass/src/tcltkgrass/README.old:                      ASCII English text
-> old: image/png for grass6 old: text/plain for grass5
pat: image/x-coreldrawpattern
grass/src/paint/Drivers/versatec/patterns/ce3200.pat: ASCII text
-> text/plain
patch: text/x-diff
pdf: application/pdf
pl: text/x-perl
pm: text/x-perl
pbm: image/x-portable-bitmap
grass/src/raster/r.in.pbm:            directory
grass/src/tcltkgrass/module/r.in.pbm: ASCII text
-> text/plain
pgm: image/x-portable-grayma
grass/src/raster/r.in.pgm:            directory
grass/src/tcltkgrass/module/r.in.pgm: ASCII text
-> text/plain
png: image/png
ppm: image/x-portable-pixmap
grass/src/paint/Programs/p.ppm:                          directory
grass/src/general/g.help/help/17.manual/Help.pages/p.ppm: ASCII text
grass/src/raster/r.out.ppm:                              directory
grass/src/raster/r.in.ppm:                                directory
grass/src/raster/r.in.ppm/trees24bit.ppm:                Netpbm PPM "rawbits" image data
grass/src/raster/r.in.ppm/image24bit.ppm:                Netpbm PPM "rawbits" image data
grass/src/tcltkgrass/module/r.out.ppm:                    ASCII text
grass/src/tcltkgrass/module/r.in.ppm:                    ASCII English text
grass/src.contrib/SCS/paint/Drivers/grey.ppm:            directory
grass/locale/ru/tcltkgrass/module/r.out.ppm:              ISO-8859 text
grass/locale/ru/tcltkgrass/module/r.in.ppm:              ISO-8859 text
-> text/plain / image/x-portable-pixmap
ps: application/postscript
py: text/x-python
raw: application/octet-stream
grass/src/mapdev/v.in.arc/testdata/f1.txt.raw:                ASCII text
grass/src/mapdev/v.in.arc/testdata/f2.txt.raw:                ASCII text
grass6/imagery/i.atcorr/test_suite/ETM4_400x400.raw:          data
grass6/imagery/i.atcorr/test_suite/ETM4_400x400_atms_corr.raw: data
-> text/plain for grass5
rgb: image/x-rgb
grass/src/display/d.rgb:                                      directory
grass/src/libes/g3d/snap.rgb:                                SGI image data, RLE, 3-D, 573 x 372, 3 channels
grass/src/general/g.help/help/17.manual/Help.pages/d.rgb:    ASCII English text
grass/src/general/g.help/help/17.manual/Help.pages/i.his.rgb: ASCII text
grass/src/general/g.help/help/17.manual/Help.pages/r.his.rgb: ASCII text
grass/src/tcltkgrass/module/d.rgb:                            ASCII text
grass/src/tcltkgrass/module/i.his.rgb:                        ASCII text
grass/src.contrib/GMSL/NVIZ2.2/TOGL/apps/ben.rgb:            SGI image data, RLE, 3-D, 133 x 158, 3 channels
grass/locale/ru/tcltkgrass/module/d.rgb:                      ISO-8859 text
grass/locale/ru/tcltkgrass/module/i.his.rgb:                  ISO-8859 text
grass/unused/tcltkgrass/module/d.rgb:                        ASCII English text
grass/unused/tcltkgrass/module/i.his.rgb:                    ASCII English text
grass6/lib/g3d/snap.rgb:                                      SGI image data, RLE, 3-D, 573 x 372, 3 channels
grass6/display/d.rgb:                                        directory
grass6/scripts/i.landsat.rgb:                                directory
grass6/scripts/i.landsat.rgb/i.landsat.rgb:                  Bourne shell script text executable
grass6/imagery/i.his.rgb:                                    directory
-> text/plain / image/x-rgb
ros: chemical/x-rosdal
grass/src/raster/wildfire/src/r.ros: directory
grass/src/tcltkgrass/module/r.ros:  ASCII English text
grass6/raster/wildfire/r.ros:        directory
-> text/plain
rtf: application/rtf
sh: text/x-sh
? src: application/x-wais-source
grass6/lib/init/grass-run.src:        POSIX shell script text executable
grass6/lib/init/grass.src:            POSIX shell script text executable
grass6/binaryInstall.src:            POSIX shell script text executable
grass/src/general/init/grass.src:    POSIX shell script text executable
grass/src/tcltkgrass/gis_set.tcl.src: ASCII English text
grass/binaryInstall.src:              POSIX shell script text executable
-> src: text/plain
? t: application/x-troff
grass6/swig/perl/t/R_slope_aspect.t: ASCII English text
-> t: text/plain
tcl: text/x-tcl
tex: text/x-tex
tgz: application/x-gtar
tiff: image/tiff
txt: text/plain
xbm: image/x-xbitmap
? xyz: chemical/x-xyz
grass6/scripts/r.out.xyz:                        setgid directory
grass6/scripts/r.out.xyz/r.out.xyz:              POSIX shell script text executable
grass6/raster/r.in.xyz:                          setgid directory
grass/src/raster/r.out.xyz:                      setgid directory
grass/src/tcltkgrass/module/r.out.xyz:          ASCII text
-> xyz: text/plain
# Instead, build a props file for use with the '--auto-props=FILE' option.
echo "[auto-props]" > propsfile-g5
sed -e 's%: \(text/.*\)% = svn:eol-style=native;svn:keywords=Author Date Id Revision;svn:mime-type=\1%' \
-e 's/^/*./' -e 's/: / = svn:mime-type=/' step6-g5 >> propsfile-g5
echo "[auto-props]" > propsfile-g6
sed -e 's%: \(text/.*\)% = svn:eol-style=native;svn:keywords=Author Date Id Revision;svn:mime-type=\1%' \
-e 's/^/*./' -e 's/: / = svn:mime-type=/' step6-g6 >> propsfile-g6
Remove mime-type:text/*
cat propsfile-g6 | sed -e 's/svn:mime-type=text\/.*//g' > propsfile-g6-1
cat propsfile-g5 | sed -e 's/svn:mime-type=text\/.*//g' > propsfile-g5-1
#Finish the generation of propsfile
sed -re 's/(.*\.([^.]*$))/\2 \1/' -e 's/^([^.]*)$/\1 \1/' step5-g6 | sort | \
sed -e 's/^[^ ]* //' -e 's%$% = svn:eol-style=native;svn:keywords=Author Date Id Revision%' >> propsfile-g6-1
#Finish the generation of propsfile
sed -re 's/(.*\.([^.]*$))/\2 \1/' -e 's/^([^.]*)$/\1 \1/' step5-g5 | sort | \
sed -e 's/^[^ ]* //' -e 's%$% = svn:eol-style=native;svn:keywords=Author Date Id Revision%' >> propsfile-g5-1
Files are available [http://josef.fsv.cvut.cz/~landa/grass-cvs2svn here].


== External links ==
== External links ==

Revision as of 23:02, 5 December 2007

This page contains notes related to GRASS code migration (planned) from CVS to SVN.

Basic

  • The SVN command line interface is just like CVS, many tasks are identical- just change the program name from cvs to svn.

Gotchas

  • cvs2svn is known to break binary files (images) which were not imported into the CVS with the -kb flag. Luckily Glynn fixed most of these some months ago.
  • Files using keyword substitution, such as $Date$ in the description.html files, will have to have support for that enabled manually, once per file (or write a find routine with | xargs svn ...).
$ svn propset svn:keywords "Date" filename.txt
$ svn commit

or

$ find . -name '*.c' | xargs svn propset svn:keywords "Date" 
$ find . -name '*.html' | xargs svn propset svn:keywords "Date" 
$ svn commit
  • how to maintain timestamps of files? We want to keep the last modification date, not the date of local download

GRASS CVS repository structure

/grass-cvs
|
+---/CVSROOT
|
+---/grass
|
+---/grass51
|
+---/grass6
|
+---/grass_doc
|
+---/libgrass
|
+---/newsletter
|
+---/programgrass50
|
+---/web



GRASS OSGeo SVN repository structure

Scenario 2

Scripts:

./grass-cvs2svn-base.sh
./grass-cvs2svn-s2.sh
./grass-cvs2svn-wxpython.sh

grass-svn2

GRASS 7 development notes


External links

SVN hosting

There are two main options to host the new SVN repository.