| author | kergoth <kergoth> | 2002-01-25 22:14:26 (UTC) |
|---|---|---|
| committer | kergoth <kergoth> | 2002-01-25 22:14:26 (UTC) |
| commit | 15318cad33835e4e2dc620d033e43cd930676cdd (patch) (side-by-side diff) | |
| tree | c2fa0399a2c47fda8e2cd0092c73a809d17f68eb /rsync | |
| download | opie-15318cad33835e4e2dc620d033e43cd930676cdd.zip opie-15318cad33835e4e2dc620d033e43cd930676cdd.tar.gz opie-15318cad33835e4e2dc620d033e43cd930676cdd.tar.bz2 | |
Initial revision
55 files changed, 8259 insertions, 0 deletions
diff --git a/rsync/AUTHORS b/rsync/AUTHORS new file mode 100644 index 0000000..f11fc44 --- a/dev/null +++ b/rsync/AUTHORS @@ -0,0 +1,2 @@ +Martin Pool <mbp@samba.org> +Andrew Tridgell <tridge@samba.org> diff --git a/rsync/COPYING b/rsync/COPYING new file mode 100644 index 0000000..c4792dd --- a/dev/null +++ b/rsync/COPYING @@ -0,0 +1,515 @@ + + GNU LESSER GENERAL PUBLIC LICENSE + Version 2.1, February 1999 + + Copyright (C) 1991, 1999 Free Software Foundation, Inc. + 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + +[This is the first released version of the Lesser GPL. It also counts + as the successor of the GNU Library Public License, version 2, hence + the version number 2.1.] + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +Licenses are intended to guarantee your freedom to share and change +free software--to make sure the software is free for all its users. + + This license, the Lesser General Public License, applies to some +specially designated software packages--typically libraries--of the +Free Software Foundation and other authors who decide to use it. You +can use it too, but we suggest you first think carefully about whether +this license or the ordinary General Public License is the better +strategy to use in any particular case, based on the explanations +below. + + When we speak of free software, we are referring to freedom of use, +not price. Our General Public Licenses are designed to make sure that +you have the freedom to distribute copies of free software (and charge +for this service if you wish); that you receive source code or can get +it if you want it; that you can change the software and use pieces of +it in new free programs; and that you are informed that you can do +these things. + + To protect your rights, we need to make restrictions that forbid +distributors to deny you these rights or to ask you to surrender these +rights. These restrictions translate to certain responsibilities for +you if you distribute copies of the library or if you modify it. + + For example, if you distribute copies of the library, whether gratis +or for a fee, you must give the recipients all the rights that we gave +you. You must make sure that they, too, receive or can get the source +code. If you link other code with the library, you must provide +complete object files to the recipients, so that they can relink them +with the library after making changes to the library and recompiling +it. And you must show them these terms so they know their rights. + + We protect your rights with a two-step method: (1) we copyright the +library, and (2) we offer you this license, which gives you legal +permission to copy, distribute and/or modify the library. + + To protect each distributor, we want to make it very clear that +there is no warranty for the free library. Also, if the library is +modified by someone else and passed on, the recipients should know +that what they have is not the original version, so that the original +author's reputation will not be affected by problems that might be +introduced by others. +^L + Finally, software patents pose a constant threat to the existence of +any free program. We wish to make sure that a company cannot +effectively restrict the users of a free program by obtaining a +restrictive license from a patent holder. Therefore, we insist that +any patent license obtained for a version of the library must be +consistent with the full freedom of use specified in this license. + + Most GNU software, including some libraries, is covered by the +ordinary GNU General Public License. This license, the GNU Lesser +General Public License, applies to certain designated libraries, and +is quite different from the ordinary General Public License. We use +this license for certain libraries in order to permit linking those +libraries into non-free programs. + + When a program is linked with a library, whether statically or using +a shared library, the combination of the two is legally speaking a +combined work, a derivative of the original library. The ordinary +General Public License therefore permits such linking only if the +entire combination fits its criteria of freedom. The Lesser General +Public License permits more lax criteria for linking other code with +the library. + + We call this license the "Lesser" General Public License because it +does Less to protect the user's freedom than the ordinary General +Public License. It also provides other free software developers Less +of an advantage over competing non-free programs. These disadvantages +are the reason we use the ordinary General Public License for many +libraries. However, the Lesser license provides advantages in certain +special circumstances. + + For example, on rare occasions, there may be a special need to +encourage the widest possible use of a certain library, so that it +becomes +a de-facto standard. To achieve this, non-free programs must be +allowed to use the library. A more frequent case is that a free +library does the same job as widely used non-free libraries. In this +case, there is little to gain by limiting the free library to free +software only, so we use the Lesser General Public License. + + In other cases, permission to use a particular library in non-free +programs enables a greater number of people to use a large body of +free software. For example, permission to use the GNU C Library in +non-free programs enables many more people to use the whole GNU +operating system, as well as its variant, the GNU/Linux operating +system. + + Although the Lesser General Public License is Less protective of the +users' freedom, it does ensure that the user of a program that is +linked with the Library has the freedom and the wherewithal to run +that program using a modified version of the Library. + + The precise terms and conditions for copying, distribution and +modification follow. Pay close attention to the difference between a +"work based on the library" and a "work that uses the library". The +former contains code derived from the library, whereas the latter must +be combined with the library in order to run. +^L + GNU LESSER GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License Agreement applies to any software library or other +program which contains a notice placed by the copyright holder or +other authorized party saying it may be distributed under the terms of +this Lesser General Public License (also called "this License"). +Each licensee is addressed as "you". + + A "library" means a collection of software functions and/or data +prepared so as to be conveniently linked with application programs +(which use some of those functions and data) to form executables. + + The "Library", below, refers to any such software library or work +which has been distributed under these terms. A "work based on the +Library" means either the Library or any derivative work under +copyright law: that is to say, a work containing the Library or a +portion of it, either verbatim or with modifications and/or translated +straightforwardly into another language. (Hereinafter, translation is +included without limitation in the term "modification".) + + "Source code" for a work means the preferred form of the work for +making modifications to it. For a library, complete source code means +all the source code for all modules it contains, plus any associated +interface definition files, plus the scripts used to control +compilation +and installation of the library. + + Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running a program using the Library is not restricted, and output from +such a program is covered only if its contents constitute a work based +on the Library (independent of the use of the Library in a tool for +writing it). Whether that is true depends on what the Library does +and what the program that uses the Library does. + + 1. You may copy and distribute verbatim copies of the Library's +complete source code as you receive it, in any medium, provided that +you conspicuously and appropriately publish on each copy an +appropriate copyright notice and disclaimer of warranty; keep intact +all the notices that refer to this License and to the absence of any +warranty; and distribute a copy of this License along with the +Library. + + You may charge a fee for the physical act of transferring a copy, +and you may at your option offer warranty protection in exchange for a +fee. + + 2. You may modify your copy or copies of the Library or any portion +of it, thus forming a work based on the Library, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) The modified work must itself be a software library. + + b) You must cause the files modified to carry prominent notices + stating that you changed the files and the date of any change. + + c) You must cause the whole of the work to be licensed at no + charge to all third parties under the terms of this License. + + d) If a facility in the modified Library refers to a function or a + table of data to be supplied by an application program that uses + the facility, other than as an argument passed when the facility + is invoked, then you must make a good faith effort to ensure that, + in the event an application does not supply such function or + table, the facility still operates, and performs whatever part of + its purpose remains meaningful. + + (For example, a function in a library to compute square roots has + a purpose that is entirely well-defined independent of the + application. Therefore, Subsection 2d requires that any + application-supplied function or table used by this function must + be optional: if the application does not supply it, the square + root function must still compute square roots.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Library, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Library, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote +it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Library. + +In addition, mere aggregation of another work not based on the Library +with the Library (or with a work based on the Library) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may opt to apply the terms of the ordinary GNU General Public +License instead of this License to a given copy of the Library. To do +this, you must alter all the notices that refer to this License, so +that they refer to the ordinary GNU General Public License, version 2, +instead of to this License. (If a newer version than version 2 of the +ordinary GNU General Public License has appeared, then you can specify +that version instead if you wish.) Do not make any other change in +these notices. +^L + Once this change is made in a given copy, it is irreversible for +that copy, so the ordinary GNU General Public License applies to all +subsequent copies and derivative works made from that copy. + + This option is useful when you wish to copy part of the code of +the Library into a program that is not a library. + + 4. You may copy and distribute the Library (or a portion or +derivative of it, under Section 2) in object code or executable form +under the terms of Sections 1 and 2 above provided that you accompany +it with the complete corresponding machine-readable source code, which +must be distributed under the terms of Sections 1 and 2 above on a +medium customarily used for software interchange. + + If distribution of object code is made by offering access to copy +from a designated place, then offering equivalent access to copy the +source code from the same place satisfies the requirement to +distribute the source code, even though third parties are not +compelled to copy the source along with the object code. + + 5. A program that contains no derivative of any portion of the +Library, but is designed to work with the Library by being compiled or +linked with it, is called a "work that uses the Library". Such a +work, in isolation, is not a derivative work of the Library, and +therefore falls outside the scope of this License. + + However, linking a "work that uses the Library" with the Library +creates an executable that is a derivative of the Library (because it +contains portions of the Library), rather than a "work that uses the +library". The executable is therefore covered by this License. +Section 6 states terms for distribution of such executables. + + When a "work that uses the Library" uses material from a header file +that is part of the Library, the object code for the work may be a +derivative work of the Library even though the source code is not. +Whether this is true is especially significant if the work can be +linked without the Library, or if the work is itself a library. The +threshold for this to be true is not precisely defined by law. + + If such an object file uses only numerical parameters, data +structure layouts and accessors, and small macros and small inline +functions (ten lines or less in length), then the use of the object +file is unrestricted, regardless of whether it is legally a derivative +work. (Executables containing this object code plus portions of the +Library will still fall under Section 6.) + + Otherwise, if the work is a derivative of the Library, you may +distribute the object code for the work under the terms of Section 6. +Any executables containing that work also fall under Section 6, +whether or not they are linked directly with the Library itself. +^L + 6. As an exception to the Sections above, you may also combine or +link a "work that uses the Library" with the Library to produce a +work containing portions of the Library, and distribute that work +under terms of your choice, provided that the terms permit +modification of the work for the customer's own use and reverse +engineering for debugging such modifications. + + You must give prominent notice with each copy of the work that the +Library is used in it and that the Library and its use are covered by +this License. You must supply a copy of this License. If the work +during execution displays copyright notices, you must include the +copyright notice for the Library among them, as well as a reference +directing the user to the copy of this License. Also, you must do one +of these things: + + a) Accompany the work with the complete corresponding + machine-readable source code for the Library including whatever + changes were used in the work (which must be distributed under + Sections 1 and 2 above); and, if the work is an executable linked + with the Library, with the complete machine-readable "work that + uses the Library", as object code and/or source code, so that the + user can modify the Library and then relink to produce a modified + executable containing the modified Library. (It is understood + that the user who changes the contents of definitions files in the + Library will not necessarily be able to recompile the application + to use the modified definitions.) + + b) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (1) uses at run time a + copy of the library already present on the user's computer system, + rather than copying library functions into the executable, and (2) + will operate properly with a modified version of the library, if + the user installs one, as long as the modified version is + interface-compatible with the version that the work was made with. + + c) Accompany the work with a written offer, valid for at + least three years, to give the same user the materials + specified in Subsection 6a, above, for a charge no more + than the cost of performing this distribution. + + d) If distribution of the work is made by offering access to copy + from a designated place, offer equivalent access to copy the above + specified materials from the same place. + + e) Verify that the user has already received a copy of these + materials or that you have already sent this user a copy. + + For an executable, the required form of the "work that uses the +Library" must include any data and utility programs needed for +reproducing the executable from it. However, as a special exception, +the materials to be distributed need not include anything that is +normally distributed (in either source or binary form) with the major +components (compiler, kernel, and so on) of the operating system on +which the executable runs, unless that component itself accompanies +the executable. + + It may happen that this requirement contradicts the license +restrictions of other proprietary libraries that do not normally +accompany the operating system. Such a contradiction means you cannot +use both them and the Library together in an executable that you +distribute. +^L + 7. You may place library facilities that are a work based on the +Library side-by-side in a single library together with other library +facilities not covered by this License, and distribute such a combined +library, provided that the separate distribution of the work based on +the Library and of the other library facilities is otherwise +permitted, and provided that you do these two things: + + a) Accompany the combined library with a copy of the same work + based on the Library, uncombined with any other library + facilities. This must be distributed under the terms of the + Sections above. + + b) Give prominent notice with the combined library of the fact + that part of it is a work based on the Library, and explaining + where to find the accompanying uncombined form of the same work. + + 8. You may not copy, modify, sublicense, link with, or distribute +the Library except as expressly provided under this License. Any +attempt otherwise to copy, modify, sublicense, link with, or +distribute the Library is void, and will automatically terminate your +rights under this License. However, parties who have received copies, +or rights, from you under this License will not have their licenses +terminated so long as such parties remain in full compliance. + + 9. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Library or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Library (or any work based on the +Library), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Library or works based on it. + + 10. Each time you redistribute the Library (or any work based on the +Library), the recipient automatically receives a license from the +original licensor to copy, distribute, link with or modify the Library +subject to these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties with +this License. +^L + 11. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Library at all. For example, if a patent +license would not permit royalty-free redistribution of the Library by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Library. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply, and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 12. If the distribution and/or use of the Library is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Library under this License +may add an explicit geographical distribution limitation excluding those +countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 13. The Free Software Foundation may publish revised and/or new +versions of the Lesser General Public License from time to time. +Such new versions will be similar in spirit to the present version, +but may differ in detail to address new problems or concerns. + +Each version is given a distinguishing version number. If the Library +specifies a version number of this License which applies to it and +"any later version", you have the option of following the terms and +conditions either of that version or of any later version published by +the Free Software Foundation. If the Library does not specify a +license version number, you may choose any version ever published by +the Free Software Foundation. +^L + 14. If you wish to incorporate parts of the Library into other free +programs whose distribution conditions are incompatible with these, +write to the author to ask for permission. For software which is +copyrighted by the Free Software Foundation, write to the Free +Software Foundation; we sometimes make exceptions for this. Our +decision will be guided by the two goals of preserving the free status +of all derivatives of our free software and of promoting the sharing +and reuse of software generally. + + NO WARRANTY + + 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO +WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. +EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR +OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY +KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE +LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME +THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN +WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY +AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU +FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR +CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE +LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING +RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A +FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF +SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH +DAMAGES. + + END OF TERMS AND CONDITIONS +^L + How to Apply These Terms to Your New Libraries + + If you develop a new library, and you want it to be of the greatest +possible use to the public, we recommend making it free software that +everyone can redistribute and change. You can do so by permitting +redistribution under these terms (or, alternatively, under the terms +of the ordinary General Public License). + + To apply these terms, attach the following notices to the library. +It is safest to attach them to the start of each source file to most +effectively convey the exclusion of warranty; and each file should +have at least the "copyright" line and a pointer to where the full +notice is found. + + + <one line to give the library's name and a brief idea of what it +does.> + Copyright (C) <year> <name of author> + + This library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2 of the License, or (at your option) any later version. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with this library; if not, write to the Free Software + Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +Also add information on how to contact you by electronic and paper +mail. + +You should also get your employer (if you work as a programmer) or +your +school, if any, to sign a "copyright disclaimer" for the library, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the + library `Frob' (a library for tweaking knobs) written by James +Random Hacker. + + <signature of Ty Coon>, 1 April 1990 + Ty Coon, President of Vice + +That's all there is to it! + + diff --git a/rsync/Makefile.in b/rsync/Makefile.in new file mode 100644 index 0000000..50e7b3d --- a/dev/null +++ b/rsync/Makefile.in @@ -0,0 +1,391 @@ +############################################################################# + +####### Compiler, tools and options + +CXX = $(SYSCONF_CXX) $(QT_CXX_MT) +CXXFLAGS= $(SYSCONF_CXXFLAGS_QT) $(SYSCONF_CXXFLAGS) $(SYSCONF_CXXFLAGS_LIB) +CC = $(SYSCONF_CC) $(QT_C_MT) +CFLAGS = $(SYSCONF_CFLAGS) $(SYSCONF_CFLAGS_LIB) +INCPATH = -I. +LFLAGS = $(SYSCONF_LFLAGS_QT) $(SYSCONF_RPATH_QT) $(SYSCONF_LFLAGS) $(QT_LFLAGS_MT) +LIBS = $(SUBLIBS) $(SYSCONF_LIBS_QT) $(SYSCONF_LIBS_QTAPP) +MOC = $(SYSCONF_MOC) +UIC = $(SYSCONF_UIC) + +####### Target + +DESTDIR = $(QTDIR)/lib$(PROJMAK)/ +VER_MAJ = 1 +VER_MIN = 0 +VER_PATCH = 0 +TARGET = rsync +TARGET1 = lib$(TARGET).so.$(VER_MAJ) + +####### Files + +HEADERS = buf.h \ + checksum.h \ + command.h \ + emit.h \ + fileutil.h \ + job.h \ + netint.h \ + protocol.h \ + prototab.h \ + rsync.h \ + search.h \ + stream.h \ + sumset.h \ + trace.h \ + types.h \ + util.h \ + whole.h \ + config.h +SOURCES = base64.c \ + buf.c \ + checksum.c \ + command.c \ + delta.c \ + emit.c \ + fileutil.c \ + hex.c \ + job.c \ + mdfour.c \ + mksum.c \ + msg.c \ + netint.c \ + patch.c \ + prototab.c \ + readsums.c \ + scoop.c \ + search.c \ + stats.c \ + stream.c \ + sumset.c \ + trace.c \ + tube.c \ + util.c \ + version.c \ + whole.c +OBJECTS = base64.o \ + buf.o \ + checksum.o \ + command.o \ + delta.o \ + emit.o \ + fileutil.o \ + hex.o \ + job.o \ + mdfour.o \ + mksum.o \ + msg.o \ + netint.o \ + patch.o \ + prototab.o \ + readsums.o \ + scoop.o \ + search.o \ + stats.o \ + stream.o \ + sumset.o \ + trace.o \ + tube.o \ + util.o \ + version.o \ + whole.o +INTERFACES = +UICDECLS = +UICIMPLS = +SRCMOC = +OBJMOC = + + +####### Implicit rules + +.SUFFIXES: .cpp .cxx .cc .C .c + +.cpp.o: + $(CXX) -c $(CXXFLAGS) $(INCPATH) -o $@ $< + +.cxx.o: + $(CXX) -c $(CXXFLAGS) $(INCPATH) -o $@ $< + +.cc.o: + $(CXX) -c $(CXXFLAGS) $(INCPATH) -o $@ $< + +.C.o: + $(CXX) -c $(CXXFLAGS) $(INCPATH) -o $@ $< + +.c.o: + $(CC) -c $(CFLAGS) $(INCPATH) -o $@ $< + +####### Build rules + + +all: $(DESTDIR)$(SYSCONF_LINK_TARGET) + +$(DESTDIR)$(SYSCONF_LINK_TARGET): $(UICDECLS) $(OBJECTS) $(OBJMOC) $(SUBLIBS) + $(SYSCONF_LINK_LIB) + +moc: $(SRCMOC) + +tmake: Makefile.in + +Makefile.in: rsync.pro + tmake rsync.pro -o Makefile.in + +clean: + -rm -f $(OBJECTS) $(OBJMOC) $(SRCMOC) $(UICIMPLS) $(UICDECLS) + -rm -f *~ core + -rm -f allmoc.cpp + +####### Extension Modules + +listpromodules: + @echo + +listallmodules: + @echo + +listaddonpromodules: + @echo + +listaddonentmodules: + @echo + + +REQUIRES= + +####### Sub-libraries + + +###### Combined headers + + + +####### Compile + +base64.o: base64.c \ + config.h \ + config_linux.h \ + rsync.h + +buf.o: buf.c \ + config.h \ + config_linux.h \ + rsync.h \ + trace.h \ + buf.h \ + util.h + +checksum.o: checksum.c \ + config.h \ + config_linux.h \ + rsync.h \ + checksum.h + +command.o: command.c \ + config.h \ + config_linux.h \ + rsync.h \ + command.h + +delta.o: delta.c \ + config.h \ + config_linux.h \ + rsync.h \ + emit.h \ + stream.h \ + util.h \ + sumset.h \ + job.h \ + trace.h \ + checksum.h \ + search.h \ + types.h + +emit.o: emit.c \ + config.h \ + config_linux.h \ + rsync.h \ + command.h \ + protocol.h \ + trace.h \ + emit.h \ + prototab.h \ + netint.h \ + sumset.h \ + job.h + +fileutil.o: fileutil.c \ + config.h \ + config_linux.h \ + rsync.h \ + fileutil.h \ + trace.h + +hex.o: hex.c \ + config.h \ + config_linux.h \ + rsync.h + +job.o: job.c \ + config.h \ + config_linux.h \ + rsync.h \ + stream.h \ + util.h \ + sumset.h \ + job.h \ + trace.h + +mdfour.o: mdfour.c \ + config.h \ + config_linux.h \ + rsync.h \ + trace.h \ + types.h + +mksum.o: mksum.c \ + config.h \ + config_linux.h \ + rsync.h \ + stream.h \ + util.h \ + sumset.h \ + job.h \ + protocol.h \ + netint.h \ + trace.h \ + checksum.h + +msg.o: msg.c \ + config.h \ + config_linux.h \ + rsync.h + +netint.o: netint.c \ + config.h \ + config_linux.h \ + rsync.h \ + job.h \ + netint.h \ + trace.h \ + stream.h + +patch.o: patch.c \ + config.h \ + config_linux.h \ + rsync.h \ + util.h \ + trace.h \ + protocol.h \ + netint.h \ + command.h \ + sumset.h \ + prototab.h \ + stream.h \ + job.h + +prototab.o: prototab.c \ + config.h \ + config_linux.h \ + rsync.h \ + protocol.h \ + command.h \ + prototab.h + +readsums.o: readsums.c \ + config.h \ + config_linux.h \ + rsync.h \ + sumset.h \ + job.h \ + trace.h \ + netint.h \ + protocol.h \ + util.h \ + stream.h + +scoop.o: scoop.c \ + config.h \ + config_linux.h \ + rsync.h \ + job.h \ + stream.h \ + trace.h \ + util.h + +search.o: search.c \ + config.h \ + config_linux.h \ + rsync.h \ + trace.h \ + util.h \ + sumset.h \ + search.h \ + checksum.h + +stats.o: stats.c \ + config.h \ + config_linux.h \ + rsync.h \ + trace.h + +stream.o: stream.c \ + config.h \ + config_linux.h \ + rsync.h \ + stream.h \ + util.h \ + trace.h + +sumset.o: sumset.c \ + config.h \ + config_linux.h \ + rsync.h \ + sumset.h \ + util.h \ + trace.h + +trace.o: trace.c \ + config.h \ + config_linux.h \ + rsync.h \ + util.h \ + trace.h + +tube.o: tube.c \ + config.h \ + config_linux.h \ + rsync.h \ + trace.h \ + util.h \ + job.h \ + stream.h + +util.o: util.c \ + config.h \ + config_linux.h \ + util.h \ + rsync.h \ + trace.h + +version.o: version.c \ + config.h \ + config_linux.h \ + rsync.h + +whole.o: whole.c \ + config.h \ + config_linux.h \ + rsync.h \ + trace.h \ + fileutil.h \ + sumset.h \ + job.h \ + buf.h \ + whole.h \ + util.h + + diff --git a/rsync/README b/rsync/README new file mode 100644 index 0000000..02df92a --- a/dev/null +++ b/rsync/README @@ -0,0 +1,62 @@ +librsync is the next generation of librsync, and provides flexible +checksum-based differencing. The main application at the moment in +rproxy, but the library should eventually be generally useful. + + http://linuxcare.com.au/rproxy/ + +This library was previously known as libhsync up to version 0.9.0. + +To use anonymous CVS, see the file README.CVS in this directory. + +>> Requirements + +To build librsync: + + * A C compiler and appropriate headers and libraries + + * Make + + * popt -- command line parsing library + + Available from ftp://ftp.redhat.com/pub/redhat/code/popt + + A cut-down version of popt1.5 is included and will be used + automatically if there is no popt library on your build host. + +>> Note for RedHat 7.0 + +RedHat 7.0 (Guiness) ships with a buggy version of GCC 2.96 that +produces many warnings while compiling librsync. These are harmless +-- the library seems to work anyhow. You can avoid the warnings by +using the 'kgcc' version of the compiler: + + $ export CC=kgcc + $ ./autogen.sh + $ make all check + +>> Library Versions + +librsync uses the GNU libtool library versioning system, so the +filename does not correspond to the librsync release. To show the +library release and version, use the librsyncinfo tool. + +>> Platforms + +librsync/rproxy is known to run on: + +GNU Linux Debian 2.2 x86 + +SUNWspro: (use -v for more warnings) + +mips-sgi-irix6.5: works, but you must use GNU Make rather than the +default SGI Make. I used gcc. + +>> API Documentation + +librsync contains markup for automatic API documentation generation +using the Doxygen tool: + + http://www.doxygen.org/ + +$Id$ + diff --git a/rsync/THANKS b/rsync/THANKS new file mode 100644 index 0000000..2e604e7 --- a/dev/null +++ b/rsync/THANKS @@ -0,0 +1,21 @@ + -*- text -*- + +Andrew Tridgell and Paulus Mackerras started this whole mess. Luke +Leighton was a tremendous help in sorting out the combined +encoding/signature algorithm. + +Thanks to Linuxcare, Inc, <http://linuxcare.com/> for their support of +this project. + +Neale Banks <neale@lowendale.com.au>: the first known user outside of +OzLabs, and helped keep me honest. + +Paul `Rusty' Russell <rusty@linuxcare.com> +Andrew Tridgell <tridge@samba.org> +Paulus Mackerras <paulus@linuxcare.com> +Peter Barker <pbarker@samba.org> +Neale Banks <neale@lowendale.com.au> +Luke Kenneth Casson Leighton <lkcl@samba.org> +Tim Potter <tpot@linuxcare.com.au> +Hugh Blemings <hugh@linuxcare.com.au> +David Gibson <dgibson@linuxcare.com.au> diff --git a/rsync/acconfig.h b/rsync/acconfig.h new file mode 100644 index 0000000..94083c7 --- a/dev/null +++ b/rsync/acconfig.h @@ -0,0 +1,55 @@ +/* acconfig.h -- hand-written definitions to eventually go into config.h */ + +/* Define this to enable trace code */ +#undef DO_RS_TRACE + +/* Version of the libtool interface. */ +#define RS_LIBVERSION "unknown" + +/* Define this if your sockaddr structure contains sin_len */ +#undef HAVE_SOCK_SIN_LEN + +/* Define this if there is a connect(2) call */ +#undef HAVE_CONNECT + +/* Define if we have an off64_t largefile type */ +#undef HAVE_OFF64_T + +/* Ask for large file support (LFS). Should always be on, even if it + * achieves nothing. */ +#undef _LARGEFILE_SOURCE +#undef _LARGEFILE64_SOURCE + +/* How many bits would you like to have in an off_t? */ +#undef _FILE_OFFSET_BITS + +/* Define to include GNU C library extensions. */ +#undef _GNU_SOURCE + +/* Define to get i18n support */ +#undef ENABLE_NLS + +/* Define if you want the suboptimal X/Open catgets implementation */ +#undef HAVE_CATGETS + +/* Define if you want the nice new GNU and Uniforum gettext system */ +#undef HAVE_GETTEXT + +/* Define if your system has the LC_MESSAGES locale category */ +#undef HAVE_LC_MESSAGES + +/* Define if you have stpcpy (copy a string and return a pointer to + * the end of the result.) */ +#undef HAVE_STPCPY + +/* GNU extension of saving argv[0] to program_invocation_short_name */ +#undef HAVE_PROGRAM_INVOCATION_NAME + +/* Canonical GNU hostname */ +#define RS_CANONICAL_HOST "unknown" + +/* Define to a replacement type if intmax_t is not a builtin, or in + sys/types.h or stdlib.h or stddef.h */ +#undef intmax_t + +/* end of acconfig.h */ diff --git a/rsync/base64.c b/rsync/base64.c new file mode 100644 index 0000000..cbe7a8c --- a/dev/null +++ b/rsync/base64.c @@ -0,0 +1,101 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +#include <config_rsync.h> + +#include <string.h> +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" + +/* + * Decode a base64 string in-place - simple and slow algorithm + * + * See RFC1521 for the specification of base64. + */ +size_t rs_unbase64(char *s) +{ + char const *b64 = + "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + int bit_offset, byte_offset, idx, i, n; + unsigned char *d = (unsigned char *) s; + char *p; + + n = i = 0; + + while (*s && (p = strchr(b64, *s))) { + idx = (int) (p - b64); + byte_offset = (i * 6) / 8; + bit_offset = (i * 6) % 8; + d[byte_offset] &= ~((1 << (8 - bit_offset)) - 1); + if (bit_offset < 3) { + d[byte_offset] |= (idx << (2 - bit_offset)); + n = byte_offset + 1; + } else { + d[byte_offset] |= (idx >> (bit_offset - 2)); + d[byte_offset + 1] = 0; + d[byte_offset + 1] |= (idx << (8 - (bit_offset - 2))) & 0xFF; + n = byte_offset + 2; + } + s++; + i++; + } + + return n; +} + +/* + * Encode a buffer as base64 - simple and slow algorithm. + */ +void +rs_base64(unsigned char const *buf, int n, char *out) +{ + char const *b64 = + "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + int bytes, i; + + /* work out how many bytes of output there are */ + bytes = ((n * 8) + 5) / 6; + + for (i = 0; i < bytes; i++) { + int byte = (i * 6) / 8; + int bit = (i * 6) % 8; + + if (bit < 3) { + if (byte >= n) + abort(); + *out = b64[(buf[byte] >> (2 - bit)) & 0x3F]; + } else { + if (byte + 1 == n) { + *out = b64[(buf[byte] << (bit - 2)) & 0x3F]; + } else { + *out = b64[(buf[byte] << (bit - 2) | + buf[byte + 1] >> (10 - bit)) & 0x3F]; + } + } + out++; + } + *out = 0; +} + diff --git a/rsync/buf.c b/rsync/buf.c new file mode 100644 index 0000000..2814583 --- a/dev/null +++ b/rsync/buf.c @@ -0,0 +1,214 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* + | Pick a window, Jimmy, you're leaving. + | -- Martin Schwenke, regularly + */ + + +/* + * buf.c -- Buffers that map between stdio file streams and librsync + * streams. As the stream consumes input and produces output, it is + * refilled from appropriate input and output FILEs. A dynamically + * allocated buffer of configurable size is used as an intermediary. + * + * TODO: Perhaps be more efficient by filling the buffer on every call + * even if not yet completely empty. Check that it's really our + * buffer, and shuffle remaining data down to the front. + * + * TODO: Perhaps expose a routine for shuffling the buffers. + */ + + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> +#include <errno.h> +#include <string.h> + +#include "rsync.h" +#include "trace.h" +#include "buf.h" +#include "util.h" + +/** + * File IO buffer sizes. + */ +int rs_inbuflen = 16000, rs_outbuflen = 16000; + + +struct rs_filebuf { + FILE *f; + char *buf; + size_t buf_len; +}; + + + +rs_filebuf_t *rs_filebuf_new(FILE *f, size_t buf_len) +{ + rs_filebuf_t *pf = rs_alloc_struct(rs_filebuf_t); + + pf->buf = rs_alloc(buf_len, "file buffer"); + pf->buf_len = buf_len; + pf->f = f; + + return pf; +} + + +void rs_filebuf_free(rs_filebuf_t *fb) +{ + if ( fb->buf ) + free ( fb->buf ); + rs_bzero(fb, sizeof *fb); + free(fb); +} + + +/* + * If the stream has no more data available, read some from F into + * BUF, and let the stream use that. On return, SEEN_EOF is true if + * the end of file has passed into the stream. + */ +rs_result rs_infilebuf_fill(rs_job_t *job, rs_buffers_t *buf, + void *opaque) +{ + int len; + rs_filebuf_t *fb = (rs_filebuf_t *) opaque; + FILE *f = fb->f; + + /* This is only allowed if either the buf has no input buffer + * yet, or that buffer could possibly be BUF. */ + if (buf->next_in != NULL) { + assert(buf->avail_in <= fb->buf_len); + assert(buf->next_in >= fb->buf); + assert(buf->next_in <= fb->buf + fb->buf_len); + } else { + assert(buf->avail_in == 0); + } + + if (buf->eof_in || (buf->eof_in = feof(f))) { + rs_trace("seen end of file on input"); + buf->eof_in = 1; + return RS_DONE; + } + + if (buf->avail_in) + /* Still some data remaining. Perhaps we should read + anyhow? */ + return RS_DONE; + + len = fread(fb->buf, 1, fb->buf_len, f); + if (len < 0) { + if (ferror(f)) { + rs_error("error filling buf from file: %s", + strerror(errno)); + return RS_IO_ERROR; + } else { + rs_error("no error bit, but got %d return when trying to read", + len); + return RS_IO_ERROR; + } + } + buf->avail_in = len; + buf->next_in = fb->buf; + + return RS_DONE; +} + + +/* + * The buf is already using BUF for an output buffer, and probably + * contains some buffered output now. Write this out to F, and reset + * the buffer cursor. + */ +rs_result rs_outfilebuf_drain(rs_job_t *job, rs_buffers_t *buf, void *opaque) +{ + int present; + rs_filebuf_t *fb = (rs_filebuf_t *) opaque; + FILE *f = fb->f; + + /* This is only allowed if either the buf has no output buffer + * yet, or that buffer could possibly be BUF. */ + if (buf->next_out == NULL) { + assert(buf->avail_out == 0); + + buf->next_out = fb->buf; + buf->avail_out = fb->buf_len; + + return RS_DONE; + } + + assert(buf->avail_out <= fb->buf_len); + assert(buf->next_out >= fb->buf); + assert(buf->next_out <= fb->buf + fb->buf_len); + + present = buf->next_out - fb->buf; + if (present > 0) { + int result; + + assert(present > 0); + + result = fwrite(fb->buf, 1, present, f); + if (present != result) { + rs_error("error draining buf to file: %s", + strerror(errno)); + return RS_IO_ERROR; + } + + buf->next_out = fb->buf; + buf->avail_out = fb->buf_len; + } + + return RS_DONE; +} + + +/** + * Default copy implementation that retrieves a part of a stdio file. + */ +rs_result rs_file_copy_cb(void *arg, off_t pos, size_t *len, void **buf) +{ + int got; + FILE *f = (FILE *) arg; + + if (fseek(f, pos, SEEK_SET)) { + rs_log(RS_LOG_ERR, "seek failed: %s", strerror(errno)); + return RS_IO_ERROR; + } + + got = fread(*buf, 1, *len, f); + if (got == -1) { + rs_error(strerror(errno)); + return RS_IO_ERROR; + } else if (got == 0) { + rs_error("unexpected eof on fd%d", fileno(f)); + return RS_INPUT_ENDED; + } else { + *len = got; + return RS_DONE; + } +} diff --git a/rsync/buf.h b/rsync/buf.h new file mode 100644 index 0000000..cb8386e --- a/dev/null +++ b/rsync/buf.h @@ -0,0 +1,31 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +typedef struct rs_filebuf rs_filebuf_t; + +rs_filebuf_t *rs_filebuf_new(FILE *f, size_t buf_len); + +void rs_filebuf_free(rs_filebuf_t *fb); + +rs_result rs_infilebuf_fill(rs_job_t *, rs_buffers_t *buf, void *fb); + +rs_result rs_outfilebuf_drain(rs_job_t *, rs_buffers_t *, void *fb); diff --git a/rsync/checksum.c b/rsync/checksum.c new file mode 100644 index 0000000..eab9e46 --- a/dev/null +++ b/rsync/checksum.c @@ -0,0 +1,81 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1996 by Andrew Tridgell + * Copyright (C) 1996 by Paul Mackerras + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" +#include "checksum.h" + + +/* This can possibly be used to restart the checksum system in the + * case where we detected corruption. I'm not sure yet how to make + * this useful in librsync. */ +int checksum_seed = 0; + +/* + * A simple 32 bit checksum that can be updated from either end + * (inspired by Mark Adler's Adler-32 checksum) + */ +unsigned int rs_calc_weak_sum(void const *p, int len) +{ + int i; + unsigned s1, s2; + unsigned char const *buf = (unsigned char const *) p; + + s1 = s2 = 0; + for (i = 0; i < (len - 4); i += 4) { + s2 += 4 * (s1 + buf[i]) + 3 * buf[i + 1] + + 2 * buf[i + 2] + buf[i + 3] + 10 * RS_CHAR_OFFSET; + s1 += (buf[i + 0] + buf[i + 1] + buf[i + 2] + buf[i + 3] + + 4 * RS_CHAR_OFFSET); + } + for (; i < len; i++) { + s1 += (buf[i] + RS_CHAR_OFFSET); + s2 += s1; + } + return (s1 & 0xffff) + (s2 << 16); +} + + +/** + * Calculate and store into SUM a strong MD4 checksum of the file + * blocks seen so far. + * + * In plain rsync, the checksum is perturbed by a seed value. This is + * used when retrying a failed transmission: we've discovered that the + * hashes collided at some point, so we're going to try again with + * different hashes to see if we can get it right. (Check tridge's + * thesis for details and to see if that's correct.) + * + * Since we can't retry a web transaction I'm not sure if it's very + * useful in rproxy. + */ +void rs_calc_strong_sum(void const *buf, size_t len, rs_strong_sum_t *sum) +{ + rs_mdfour((unsigned char *) sum, buf, len); +} diff --git a/rsync/checksum.h b/rsync/checksum.h new file mode 100644 index 0000000..b9fe543 --- a/dev/null +++ b/rsync/checksum.h @@ -0,0 +1,30 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +rs_weak_sum_t rs_calc_weak_sum(void const *buf1, int len); + +void rs_calc_strong_sum(void const *buf, size_t buf_len, rs_strong_sum_t *); + +/* We should make this something other than zero to improve the + * checksum algorithm: tridge suggests a prime number. */ +#define RS_CHAR_OFFSET 31 + diff --git a/rsync/command.c b/rsync/command.c new file mode 100644 index 0000000..9d56733 --- a/dev/null +++ b/rsync/command.c @@ -0,0 +1,61 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" +#include "command.h" + +/* For debugging purposes, here are some human-readable forms. */ +struct rs_op_kind_name const rs_op_kind_names[] = { + {"END", RS_KIND_END }, + {"COPY", RS_KIND_COPY }, + {"LITERAL", RS_KIND_LITERAL }, + {"SIGNATURE", RS_KIND_SIGNATURE }, + {"CHECKSUM", RS_KIND_CHECKSUM }, + {"INVALID", RS_KIND_INVALID }, + {NULL, 0 } +}; + + +/* + * Return a human-readable name for KIND. + */ +char const * rs_op_kind_name(enum rs_op_kind kind) +{ + const struct rs_op_kind_name *k; + + for (k = rs_op_kind_names; k->kind; k++) { + if (k->kind == kind) { + return k->name; + } + } + + return NULL; +} + + diff --git a/rsync/command.h b/rsync/command.h new file mode 100644 index 0000000..43da19d --- a/dev/null +++ b/rsync/command.h @@ -0,0 +1,58 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell <tridge@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +/* + * command.h -- Types of commands present in the encoding stream. + * + * The vague idea is that eventually this file will be more abstract + * than protocol.h, but it's not clear that will ever be required. + */ + + +/** + * Classes of operation that can be present. Each may have several different + * possible representations. + */ +enum rs_op_kind { + RS_KIND_END = 1000, + RS_KIND_LITERAL, + RS_KIND_SIGNATURE, + RS_KIND_COPY, + RS_KIND_CHECKSUM, + RS_KIND_RESERVED, /* for future expansion */ + + /* This one should never occur in file streams. It's an + * internal marker for invalid commands. */ + RS_KIND_INVALID +}; + + +typedef struct rs_op_kind_name { + char const *name; + enum rs_op_kind const kind; +} rs_op_kind_name_t; + +char const * rs_op_kind_name(enum rs_op_kind); + + diff --git a/rsync/config_linux.h b/rsync/config_linux.h new file mode 100644 index 0000000..e5ff3e4 --- a/dev/null +++ b/rsync/config_linux.h @@ -0,0 +1,115 @@ +/* config.h. Generated automatically by configure. */ +/* config.h.in. Generated automatically from configure.in by autoheader 2.13. */ + +/* Define to empty if the keyword does not work. */ +/* #undef const */ + +/* Define to `long' if <sys/types.h> doesn't define. */ +/* #undef off_t */ + +/* Define to `unsigned' if <sys/types.h> doesn't define. */ +/* #undef size_t */ + +/* Define if you have the ANSI C header files. */ +#define STDC_HEADERS 1 + +/* Define this if your sockaddr structure contains sin_len */ +/* #undef HAVE_SOCK_SIN_LEN */ + +/* How many bits would you like to have in an off_t? */ +#define _FILE_OFFSET_BITS 64 + +/* Define to include GNU C library extensions. */ +#define _GNU_SOURCE 1 + +/* GNU extension of saving argv[0] to program_invocation_short_name */ +#define HAVE_PROGRAM_INVOCATION_NAME 1 + +/* Define to a replacement type if intmax_t is not a builtin, or in + sys/types.h or stdlib.h or stddef.h */ +/* #undef intmax_t */ + +/* The number of bytes in a int. */ +#define SIZEOF_INT 4 + +/* The number of bytes in a long. */ +#define SIZEOF_LONG 4 + +/* The number of bytes in a off_t. */ +#define SIZEOF_OFF_T 8 + +/* The number of bytes in a short. */ +#define SIZEOF_SHORT 2 + +/* The number of bytes in a size_t. */ +#define SIZEOF_SIZE_T 4 + +/* The number of bytes in a unsigned char. */ +#define SIZEOF_UNSIGNED_CHAR 1 + +/* The number of bytes in a unsigned int. */ +#define SIZEOF_UNSIGNED_INT 4 + +/* The number of bytes in a unsigned long. */ +#define SIZEOF_UNSIGNED_LONG 4 + +/* The number of bytes in a unsigned short. */ +#define SIZEOF_UNSIGNED_SHORT 2 + +/* Define if you have the mtrace function. */ +#define HAVE_MTRACE 1 + +/* Define if you have the snprintf function. */ +#define HAVE_SNPRINTF 1 + +/* Define if you have the strerror function. */ +#define HAVE_STRERROR 1 + +/* Define if you have the vsnprintf function. */ +#define HAVE_VSNPRINTF 1 + +/* Define if you have the <alloca.h> header file. */ +#define HAVE_ALLOCA_H 1 + +/* Define if you have the <bzlib.h> header file. */ +#define HAVE_BZLIB_H 1 + +/* Define if you have the <config.h> header file. */ +/* #undef HAVE_CONFIG_H */ + +/* Define if you have the <dlfcn.h> header file. */ +#define HAVE_DLFCN_H 1 + +/* Define if you have the <libintl.h> header file. */ +#define HAVE_LIBINTL_H 1 + +/* Define if you have the <mcheck.h> header file. */ +#define HAVE_MCHECK_H 1 + +/* Define if you have the <stdint.h> header file. */ +#define HAVE_STDINT_H 1 + +/* Define if you have the <unistd.h> header file. */ +#define HAVE_UNISTD_H 1 + +/* Define if you have the popt library (-lpopt). */ +#define HAVE_LIBPOPT 1 + +/* Name of package */ +#define PACKAGE "librsync" + +/* Version number of package */ +#define VERSION "0.9.5" + +/* Number of bits in a file offset, on hosts where this is settable. */ +#define _FILE_OFFSET_BITS 64 + +/* Define to make ftello visible on some hosts (e.g. HP-UX 10.20). */ +/* #undef _LARGEFILE_SOURCE */ + +/* Define for large files, on AIX-style hosts. */ +/* #undef _LARGE_FILES */ + +/* Define to make ftello visible on some hosts (e.g. glibc 2.1.3). */ +/* #undef _XOPEN_SOURCE */ + diff --git a/rsync/config_rsync.h b/rsync/config_rsync.h new file mode 100644 index 0000000..a646eba --- a/dev/null +++ b/rsync/config_rsync.h @@ -0,0 +1,2 @@ +#include "config_linux.h" + diff --git a/rsync/delta.c b/rsync/delta.c new file mode 100644 index 0000000..323c079 --- a/dev/null +++ b/rsync/delta.c @@ -0,0 +1,351 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* + | Let's climb to the TOP of that + | MOUNTAIN and think about STRIP + | MINING!! + */ + + +/* + * delta.c -- Generate in streaming mode an rsync delta given a set of + * signatures, and a new file. + * + * The size of blocks for signature generation is determined by the + * block size in the incoming signature. + * + * To calculate a signature, we need to be able to see at least one + * block of the new file at a time. Once we have that, we calculate + * its weak signature, and see if there is any block in the signature + * hash table that has the same weak sum. If there is one, then we + * also compute the strong sum of the new block, and cross check that. + * If they're the same, then we can assume we have a match. + * + * The final block of the file has to be handled a little differently, + * because it may be a short match. Short blocks in the signature + * don't include their length -- we just allow for the final short + * block of the file to match any block in the signature, and if they + * have the same checksum we assume they must have the same length. + * Therefore, when we emit a COPY command, we have to send it with a + * length that is the same as the block matched, and not the block + * length from the signature. + */ + +/* + * Profiling results as of v1.26, 2001-03-18: + * + * If everything matches, then we spend almost all our time in + * rs_mdfour64 and rs_weak_sum, which is unavoidable and therefore a + * good profile. + * + * If nothing matches, it is not so good. + */ + + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" +#include "emit.h" +#include "stream.h" +#include "util.h" +#include "sumset.h" +#include "job.h" +#include "trace.h" +#include "checksum.h" +#include "search.h" +#include "types.h" + + +/** + * Turn this on to make all rolling checksums be checked from scratch. + */ +int rs_roll_paranoia = 0; + + +static rs_result rs_delta_scan(rs_job_t *, rs_long_t avail_len, void *); + +static rs_result rs_delta_s_deferred_copy(rs_job_t *job); + + + +static rs_result rs_delta_s_end(rs_job_t *job) +{ + rs_emit_end_cmd(job); + return RS_DONE; +} + + +/** + * \brief Get a block of data if possible, and see if it matches. + * + * On each call, we try to process all of the input data available on + * the scoop and input buffer. + */ +static rs_result +rs_delta_s_scan(rs_job_t *job) +{ + size_t this_len, avail_len; + int is_ending; + void *inptr; + rs_result result; + + rs_job_check(job); + + avail_len = rs_scoop_total_avail(job); + this_len = job->block_len; + is_ending = job->stream->eof_in; + + /* Now, we have avail_len bytes, and we need to scan through them + * looking for a match. We'll always end up emitting exactly one + * command, either a literal or a copy, and after discovering that + * we will skip over the appropriate number of bytes. */ + if (avail_len == 0) { + if (is_ending) { + /* no more delta to do */ + job->statefn = rs_delta_s_end; + } + return RS_BLOCKED; + } + + /* must read at least one block, or give up */ + if ((avail_len < job->block_len) && !is_ending) { + /* we know we won't get it, but we have to try for a whole + * block anyhow so that it gets into the scoop. */ + rs_scoop_input(job, job->block_len); + return RS_BLOCKED; + } + + result = rs_scoop_readahead(job, avail_len, &inptr); + if (result != RS_DONE) + return result; + + return rs_delta_scan(job, avail_len, inptr); +} + + + +/** + * Scan for a matching block in the next \p avail_len bytes of input. + * + * If nonmatching data is found, then a LITERAL command will be put in + * the tube immediately. If matching data is found, then its position + * will be saved in the job, and the job state set up to write out a + * COPY command after handling the literal. + */ +static rs_result +rs_delta_scan(rs_job_t *job, rs_long_t avail_len, void *p) +{ + rs_long_t match_where; + int search_pos, end_pos; + unsigned char *inptr = (unsigned char *) p; + uint32_t s1 = job->weak_sig & 0xFFFF; + uint32_t s2 = job->weak_sig >> 16; + + /* So, we have avail_len bytes of data, and we want to look + * through it for a match at some point. It's OK if it's not at + * the start of the available input data. If we're approaching + * the end and can't get a match, then we just block and get more + * later. */ + + /* FIXME: Perhaps we should be working in signed chars for the + * rolling sum? */ + + if (job->stream->eof_in) + end_pos = avail_len - 1; + else + end_pos = avail_len - job->block_len; + + for (search_pos = 0; search_pos <= end_pos; search_pos++) { + size_t this_len = job->block_len; + + if (search_pos + this_len > avail_len) { + this_len = avail_len - search_pos; + rs_trace("block reduced to %d", this_len); + } else if (job->have_weak_sig) { + unsigned char a = inptr[search_pos + this_len - 1]; + /* roll in the newly added byte, if any */ + s1 += a + RS_CHAR_OFFSET; + s2 += s1; + + job->weak_sig = (s1 & 0xffff) | (s2 << 16); + } + + if (!job->have_weak_sig) { + rs_trace("calculate weak sum from scratch"); + job->weak_sig = rs_calc_weak_sum(inptr + search_pos, this_len); + s1 = job->weak_sig & 0xFFFF; + s2 = job->weak_sig >> 16; + job->have_weak_sig = 1; + } + + if (rs_roll_paranoia) { + rs_weak_sum_t verify = rs_calc_weak_sum(inptr + search_pos, this_len); + if (verify != job->weak_sig) { + rs_fatal("mismatch between rolled sum %#x and check %#x", + job->weak_sig, verify); + } + } + + if (rs_search_for_block(job->weak_sig, inptr + search_pos, this_len, + job->signature, &job->stats, &match_where)) { + /* So, we got a match. Cool. However, there may be + * leading unmatched data that we need to flush. Thus we + * set our statefn to be rs_delta_s_deferred_copy so that + * we can write out the command later. */ + + rs_trace("matched %.0f bytes at %.0f!", + (double) this_len, (double) match_where); + job->basis_pos = match_where; + job->basis_len = this_len; + job->statefn = rs_delta_s_deferred_copy; + job->have_weak_sig = 0; + break; + } else { + /* advance by one; roll out the byte we just moved over. */ + unsigned char a = inptr[search_pos]; + unsigned char shift = a + RS_CHAR_OFFSET; + + s1 -= shift; + s2 -= this_len * shift; + job->weak_sig = (s1 & 0xffff) | (s2 << 16); + } + } + + if (search_pos > 0) { + /* We may or may not have found a block, but we know we found + * some literal data at the start of the buffer. Therefore, + * we have to flush that out before we can continue on and + * emit the copy command or keep searching. */ + + /* FIXME: At the moment, if you call with very short buffers, + * then you will get a series of very short LITERAL commands. + * Perhaps this is what you deserve, or perhaps we should try + * to get more readahead and avoid that. */ + + /* There's some literal data at the start of this window which + * we know is not in any block. */ + rs_trace("got %d bytes of literal data", search_pos); + rs_emit_literal_cmd(job, search_pos); + rs_tube_copy(job, search_pos); + } + + return RS_RUNNING; +} + + + +static rs_result rs_delta_s_deferred_copy(rs_job_t *job) +{ + if (!job->basis_len) { + rs_log(RS_LOG_ERR, "somehow got zero basis_len"); + return RS_INTERNAL_ERROR; + } + + rs_emit_copy_cmd(job, job->basis_pos, job->basis_len); + rs_scoop_advance(job, job->basis_len); + + job->statefn = rs_delta_s_scan; + + return RS_RUNNING; +} + + +/** + * \brief State function that does a slack delta containing only + * literal data to recreate the input. + */ +static rs_result rs_delta_s_slack(rs_job_t *job) +{ + rs_buffers_t * const stream = job->stream; + size_t avail = stream->avail_in; + + if (avail) { + rs_trace("emit slack delta for %.0f available bytes", (double) avail); + rs_emit_literal_cmd(job, avail); + rs_tube_copy(job, avail); + return RS_RUNNING; + } else { + if (rs_job_input_is_ending(job)) { + job->statefn = rs_delta_s_end; + return RS_RUNNING; + } else { + return RS_BLOCKED; + } + } +} + + +/** + * State function for writing out the header of the encoding job. + */ +static rs_result rs_delta_s_header(rs_job_t *job) +{ + rs_emit_delta_header(job); + + if (job->block_len) { + if (!job->signature) { + rs_error("no signature is loaded into the job"); + return RS_PARAM_ERROR; + } + job->statefn = rs_delta_s_scan; + } else { + rs_trace("block length is zero for this delta; " + "therefore using slack deltas"); + job->statefn = rs_delta_s_slack; + } + + return RS_RUNNING; +} + + +/** + * Prepare to compute a streaming delta. + */ +rs_job_t *rs_delta_begin(rs_signature_t *sig) +{ + rs_job_t *job; + + job = rs_job_new("delta", rs_delta_s_header); + job->signature = sig; + + if ((job->block_len = sig->block_len) < 0) { + rs_log(RS_LOG_ERR, "unreasonable block_len %d in signature", + job->block_len); + return NULL; + } + + job->strong_sum_len = sig->strong_sum_len; + if (job->strong_sum_len < 0 || job->strong_sum_len > RS_MD4_LENGTH) { + rs_log(RS_LOG_ERR, "unreasonable strong_sum_len %d in signature", + job->strong_sum_len); + return NULL; + } + + return job; +} + + diff --git a/rsync/emit.c b/rsync/emit.c new file mode 100644 index 0000000..bdfc6d1 --- a/dev/null +++ b/rsync/emit.c @@ -0,0 +1,131 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- dynamic caching and delta update in HTTP + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + + /* + * [almost sobbing] They don't sleep + * anymore on the beach. They don't + * sleep on the beach anymore. + */ + +/* + * TODO: Pluggable encoding formats: + * + * - gdiff-style + * - rsync 24 + * - ed (text) + * - Delta HTTP + */ + + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" +#include "command.h" +#include "protocol.h" +#include "trace.h" +#include "emit.h" +#include "prototab.h" +#include "netint.h" +#include "sumset.h" +#include "job.h" + + +/* + * Write the magic for the start of a delta. + */ +void +rs_emit_delta_header(rs_job_t *job) +{ + rs_trace("emit DELTA magic"); + rs_squirt_n4(job, RS_DELTA_MAGIC); +} + + + +/* Write a LITERAL command. */ +void +rs_emit_literal_cmd(rs_job_t *job, int len) +{ + int cmd; + int param_len; + + switch (param_len = rs_int_len(len)) { + case 1: + cmd = RS_OP_LITERAL_N1; + break; + case 2: + cmd = RS_OP_LITERAL_N2; + break; + case 4: + cmd = RS_OP_LITERAL_N4; + break; + default: + rs_fatal("What?"); + } + + rs_trace("emit LITERAL_N%d(len=%d), cmd_byte=%#x", param_len, len, cmd); + rs_squirt_byte(job, cmd); + rs_squirt_netint(job, len, param_len); + + job->stats.lit_cmds++; + job->stats.lit_bytes += len; + job->stats.lit_cmdbytes += 1 + param_len; +} + + +/** Write a COPY command. */ +void +rs_emit_copy_cmd(rs_job_t *job, rs_long_t where, rs_long_t len) +{ + int cmd; + rs_stats_t *stats = &job->stats; + + cmd = RS_OP_COPY_N4_N4; + + rs_trace("emit COPY_N4_N4(where=%.0f, len=%.0f), cmd_byte=%#x", + (double) where, (double) len, cmd); + rs_squirt_byte(job, cmd); + rs_squirt_netint(job, where, 4); + rs_squirt_netint(job, len, 4); + + stats->copy_cmds++; + stats->copy_bytes += len; + stats->copy_cmdbytes += 1 + 4 + 4; + + /* TODO: All the stats */ +} + + +/** Write an END command. */ +void +rs_emit_end_cmd(rs_job_t *job) +{ + int cmd = RS_OP_END; + + rs_trace("emit END, cmd_byte=%#x", cmd); + rs_squirt_byte(job, cmd); +} diff --git a/rsync/emit.h b/rsync/emit.h new file mode 100644 index 0000000..96bff89 --- a/dev/null +++ b/rsync/emit.h @@ -0,0 +1,32 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- dynamic caching and delta update in HTTP + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +/* + * emit.h -- How to emit commands to the client + */ + + +void rs_emit_delta_header(rs_job_t *); +void rs_emit_literal_cmd(rs_job_t *, int len); +void rs_emit_end_cmd(rs_job_t *); +void rs_emit_copy_cmd(rs_job_t *job, rs_long_t where, rs_long_t len); diff --git a/rsync/fileutil.c b/rsync/fileutil.c new file mode 100644 index 0000000..75a6bb4 --- a/dev/null +++ b/rsync/fileutil.c @@ -0,0 +1,70 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell <tridge@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <unistd.h> +#include <stdio.h> +#include <fcntl.h> +#include <sys/file.h> +#include <string.h> +#include <errno.h> + +#include "rsync.h" +#include "fileutil.h" +#include "trace.h" + + + +/** + * \brief Open a file, with special handling for `-' or unspecified + * parameters on input and output. + * + * \param fopen-style mode string. + */ +FILE * +rs_file_open(char const *filename, char const *mode) +{ + FILE *f; + int is_write; + + is_write = mode[0] == 'w'; + + if (!filename || !strcmp("-", filename)) { + if (is_write) + return stdout; + else + return stdin; + } + + if (!(f = fopen(filename, mode))) { + rs_error("Error opening \"%s\" for %s: %s", filename, + is_write ? "write" : "read", + strerror(errno)); + exit(RS_IO_ERROR); + } + + return f; +} diff --git a/rsync/fileutil.h b/rsync/fileutil.h new file mode 100644 index 0000000..b46f7c1 --- a/dev/null +++ b/rsync/fileutil.h @@ -0,0 +1,23 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- * + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +FILE * rs_file_open(char const *filename, char const * mode); diff --git a/rsync/hex.c b/rsync/hex.c new file mode 100644 index 0000000..e10b686 --- a/dev/null +++ b/rsync/hex.c @@ -0,0 +1,46 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * rproxy -- dynamic caching and delta update in HTTP + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +#include <config_rsync.h> + +#include <assert.h> +#include <sys/types.h> +#include <inttypes.h> +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" + + +void +rs_hexify(char *to_buf, void const *from, int from_len) +{ + static const char hex_chars[] = "0123456789abcdef"; + unsigned char const *from_buf = (unsigned char const *) from; + + while (from_len-- > 0) { + *(to_buf++) = hex_chars[((*from_buf) >> 4) & 0xf]; + *(to_buf++) = hex_chars[(*from_buf) & 0xf]; + from_buf++; + } + + *to_buf = 0; +} diff --git a/rsync/job.c b/rsync/job.c new file mode 100644 index 0000000..680982d --- a/dev/null +++ b/rsync/job.c @@ -0,0 +1,251 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + + /* + | The hard, lifeless I covered up the + | warm, pulsing It; protecting and + | sheltering. + */ + +/* + * job.c -- Generic state-machine interface. The point of this is + * that we need to be able to suspend and resume processing at any + * point at which the buffers may block. We could do that using + * setjmp or similar tricks, but this is probably simpler. + * + * TODO: We have a few functions to do with reading a netint, stashing + * it somewhere, then moving into a different state. Is it worth + * writing generic functions fo r that, or would it be too confusing? + */ + + +#include <config_rsync.h> + +#include <stdlib.h> +#include <assert.h> +#include <stdio.h> + +#include "rsync.h" +#include "stream.h" +#include "util.h" +#include "sumset.h" +#include "job.h" +#include "trace.h" + + +static const int rs_job_tag = 20010225; + +static rs_result rs_job_work(rs_job_t *job, rs_buffers_t *buffers); + + +rs_job_t * rs_job_new(char const *job_name, rs_result (*statefn)(rs_job_t *)) +{ + rs_job_t *job; + + job = rs_alloc_struct(rs_job_t); + + job->job_name = job_name; + job->dogtag = rs_job_tag; + job->statefn = statefn; + + job->stats.op = job_name; + + rs_trace("start %s job", job_name); + + return job; +} + + +void rs_job_check(rs_job_t *job) +{ + assert(job->dogtag == rs_job_tag); +} + + +rs_result rs_job_free(rs_job_t *job) +{ + rs_bzero(job, sizeof *job); + free(job); + + return RS_DONE; +} + + + +static rs_result rs_job_s_complete(rs_job_t *job) +{ + rs_fatal("should not be reached"); + return RS_INTERNAL_ERROR; +} + + +static rs_result rs_job_complete(rs_job_t *job, rs_result result) +{ + rs_job_check(job); + + job->statefn = rs_job_s_complete; + job->final_result = result; + + if (result != RS_DONE) { + rs_error("%s job failed: %s", job->job_name, rs_strerror(result)); + } else { + rs_trace("%s job complete", job->job_name); + } + + if (result == RS_DONE && !rs_tube_is_idle(job)) + /* Processing is finished, but there is still some data + * waiting to get into the output buffer. */ + return RS_BLOCKED; + else + return result; +} + + +/** + * \brief Run a ::rs_job_t state machine until it blocks + * (::RS_BLOCKED), returns an error, or completes (::RS_COMPLETE). + * + * \return The ::rs_result that caused iteration to stop. + * + * \param ending True if there is no more data after what's in the + * input buffer. The final block checksum will run across whatever's + * in there, without trying to accumulate anything else. + */ +rs_result rs_job_iter(rs_job_t *job, rs_buffers_t *buffers) +{ + rs_result result; + rs_long_t orig_in, orig_out; + + orig_in = buffers->avail_in; + orig_out = buffers->avail_out; + + result = rs_job_work(job, buffers); + + if (result == RS_BLOCKED || result == RS_DONE) + if ((orig_in == buffers->avail_in) && (orig_out == buffers->avail_out) + && orig_in && orig_out) { + rs_log(RS_LOG_ERR, "internal error: job made no progress " + "[orig_in=%.0f, orig_out=%.0f, final_in=%.0f, final_out=%.0f]", + (double) orig_in, (double) orig_out, (double) buffers->avail_in, + (double) buffers->avail_out); + return RS_INTERNAL_ERROR; + } + + return result; +} + + +static rs_result +rs_job_work(rs_job_t *job, rs_buffers_t *buffers) +{ + rs_result result; + + rs_job_check(job); + + if (!buffers) { + rs_error("NULL buffer passed to rs_job_iter"); + return RS_PARAM_ERROR; + } + job->stream = buffers; + + while (1) { + result = rs_tube_catchup(job); + if (result == RS_BLOCKED) + return result; + else if (result != RS_DONE) + return rs_job_complete(job, result); + + if (job->statefn == rs_job_s_complete) { + if (rs_tube_is_idle(job)) + return RS_DONE; + else + return RS_BLOCKED; + } else { + result = job->statefn(job); + if (result == RS_RUNNING) + continue; + else if (result == RS_BLOCKED) + return result; + else + return rs_job_complete(job, result); + } + } + + /* TODO: Before returning, check that we actually made some + * progress. If not, and we're not returning an error, this is a + * bug. */ +} + + +/** + * Return pointer to statistics accumulated about this job. + */ +const rs_stats_t * +rs_job_statistics(rs_job_t *job) +{ + return &job->stats; +} + + +int +rs_job_input_is_ending(rs_job_t *job) +{ + return job->stream->eof_in; +} + + + +/** + * Actively process a job, by making callbacks to fill and empty the + * buffers until the job is done. + */ +rs_result +rs_job_drive(rs_job_t *job, rs_buffers_t *buf, + rs_driven_cb in_cb, void *in_opaque, + rs_driven_cb out_cb, void *out_opaque) +{ + rs_result result, iores; + + rs_bzero(buf, sizeof *buf); + + do { + if (!buf->eof_in && in_cb) { + iores = in_cb(job, buf, in_opaque); + if (iores != RS_DONE) + return iores; + } + + result = rs_job_iter(job, buf); + if (result != RS_DONE && result != RS_BLOCKED) + return result; + + if (out_cb) { + iores = (out_cb)(job, buf, out_opaque); + if (iores != RS_DONE) + return iores; + } + } while (result != RS_DONE); + + return result; +} + diff --git a/rsync/job.h b/rsync/job.h new file mode 100644 index 0000000..33c0cd7 --- a/dev/null +++ b/rsync/job.h @@ -0,0 +1,99 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +struct rs_job { + int dogtag; + + /** Human-readable job operation name. */ + const char *job_name; + + rs_buffers_t *stream; + + /** Callback for each processing step. */ + rs_result (*statefn)(rs_job_t *); + + /** Final result of processing job. Used by rs_job_s_failed(). */ + rs_result final_result; + + /* XXX: These next two are redundant with their equivalents in the + * signature field. Perhaps we should get rid of them, but + * they're also used in the mksum operation. */ + int block_len; + int strong_sum_len; + + /** Signature that's either being read in, or used for + * generating a delta. */ + rs_signature_t *signature; + + /** Command byte currently being processed, if any. */ + unsigned char op; + + /** If in the middle of reading a signature (rs_loadsig_s_weak()), + * or generating a delta, this contains the weak signature. */ + rs_weak_sum_t weak_sig; + + /** If generating a delta, this is true if we have a valid weak signature and + * can roll it forward. */ + int have_weak_sig; + + /** Lengths of expected parameters. */ + rs_long_t param1, param2; + + struct rs_prototab_ent const *cmd; + rs_mdfour_t output_md4; + + /** Encoding statistics. */ + rs_stats_t stats; + + /** Buffer of data left over in the scoop. Allocation is + * scoop_buf..scoop_alloc, and scoop_next[0..scoop_avail] + * contains valid data. */ + char *scoop_buf; + char *scoop_next; + size_t scoop_alloc; + size_t scoop_avail; + + /** If USED is >0, then buf contains that much write data to + * be sent out. */ + char write_buf[16]; + int write_len; + + /** If \p copy_len is >0, then that much data should be copied + * through from the input. */ + rs_long_t copy_len; + + /** Copy from the basis position. */ + rs_long_t basis_pos, basis_len; + + /** Callback used to copy data from the basis into the output. */ + rs_copy_cb *copy_cb; + void *copy_arg; +}; + + +rs_job_t * rs_job_new(const char *, rs_result (*statefn)(rs_job_t *)); + +void rs_job_check(rs_job_t *job); +const rs_stats_t *rs_job_statistics(rs_job_t *); + +int rs_job_input_is_ending(rs_job_t *job); diff --git a/rsync/mdfour.c b/rsync/mdfour.c new file mode 100644 index 0000000..4c3568d --- a/dev/null +++ b/rsync/mdfour.c @@ -0,0 +1,326 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1997-1999 by Andrew Tridgell + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +/* MD4 message digest algorithm. + * + * TODO: Perhaps use the MD4 routine from OpenSSL if it's installed. + * It's probably not worth the trouble. + * + * This was originally written by Andrew Tridgell for use in Samba. */ + +#include <config_rsync.h> + +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +#include "rsync.h" +#include "trace.h" +#include "types.h" + + +static void (*rs_mdfour_block)(rs_mdfour_t *md, void const *p) = NULL; + + +#define F(X,Y,Z) (((X)&(Y)) | ((~(X))&(Z))) +#define G(X,Y,Z) (((X)&(Y)) | ((X)&(Z)) | ((Y)&(Z))) +#define H(X,Y,Z) ((X)^(Y)^(Z)) +#define lshift(x,s) (((x)<<(s)) | ((x)>>(32-(s)))) + +#define ROUND1(a,b,c,d,k,s) a = lshift(a + F(b,c,d) + X[k], s) +#define ROUND2(a,b,c,d,k,s) a = lshift(a + G(b,c,d) + X[k] + 0x5A827999,s) +#define ROUND3(a,b,c,d,k,s) a = lshift(a + H(b,c,d) + X[k] + 0x6ED9EBA1,s) + +/** + * Update an MD4 accumulator from a 64-byte chunk. + * + * This cannot be used for the last chunk of the file, which must be + * padded and contain the file length. rs_mdfour_tail() is used for + * that. + * + * \todo Recode to be fast, and to use system integer types. Perhaps + * if we can find an mdfour implementation already on the system + * (e.g. in OpenSSL) then we should use it instead of our own? + * + * \param X A series of integer, read little-endian from the file. + */ +static void +rs_mdfour64(rs_mdfour_t * m, const void *p) +{ + uint32_t AA, BB, CC, DD; + uint32_t A, B, C, D; + const uint32_t *X = (const uint32_t *) p; + + A = m->A; + B = m->B; + C = m->C; + D = m->D; + AA = A; + BB = B; + CC = C; + DD = D; + + ROUND1(A, B, C, D, 0, 3); + ROUND1(D, A, B, C, 1, 7); + ROUND1(C, D, A, B, 2, 11); + ROUND1(B, C, D, A, 3, 19); + ROUND1(A, B, C, D, 4, 3); + ROUND1(D, A, B, C, 5, 7); + ROUND1(C, D, A, B, 6, 11); + ROUND1(B, C, D, A, 7, 19); + ROUND1(A, B, C, D, 8, 3); + ROUND1(D, A, B, C, 9, 7); + ROUND1(C, D, A, B, 10, 11); + ROUND1(B, C, D, A, 11, 19); + ROUND1(A, B, C, D, 12, 3); + ROUND1(D, A, B, C, 13, 7); + ROUND1(C, D, A, B, 14, 11); + ROUND1(B, C, D, A, 15, 19); + + ROUND2(A, B, C, D, 0, 3); + ROUND2(D, A, B, C, 4, 5); + ROUND2(C, D, A, B, 8, 9); + ROUND2(B, C, D, A, 12, 13); + ROUND2(A, B, C, D, 1, 3); + ROUND2(D, A, B, C, 5, 5); + ROUND2(C, D, A, B, 9, 9); + ROUND2(B, C, D, A, 13, 13); + ROUND2(A, B, C, D, 2, 3); + ROUND2(D, A, B, C, 6, 5); + ROUND2(C, D, A, B, 10, 9); + ROUND2(B, C, D, A, 14, 13); + ROUND2(A, B, C, D, 3, 3); + ROUND2(D, A, B, C, 7, 5); + ROUND2(C, D, A, B, 11, 9); + ROUND2(B, C, D, A, 15, 13); + + ROUND3(A, B, C, D, 0, 3); + ROUND3(D, A, B, C, 8, 9); + ROUND3(C, D, A, B, 4, 11); + ROUND3(B, C, D, A, 12, 15); + ROUND3(A, B, C, D, 2, 3); + ROUND3(D, A, B, C, 10, 9); + ROUND3(C, D, A, B, 6, 11); + ROUND3(B, C, D, A, 14, 15); + ROUND3(A, B, C, D, 1, 3); + ROUND3(D, A, B, C, 9, 9); + ROUND3(C, D, A, B, 5, 11); + ROUND3(B, C, D, A, 13, 15); + ROUND3(A, B, C, D, 3, 3); + ROUND3(D, A, B, C, 11, 9); + ROUND3(C, D, A, B, 7, 11); + ROUND3(B, C, D, A, 15, 15); + + A += AA; + B += BB; + C += CC; + D += DD; + + m->A = A; + m->B = B; + m->C = C; + m->D = D; +} + + +/* These next two routines are necessary because MD4 is specified in + * terms of little-endian int32s, but we have a byte buffer. On + * little-endian platforms, I think we can just use the buffer pointer + * directly. + * + * There are some nice endianness routines in glib, including + * assembler variants. If we ever depended on glib, then it could be + * good to use them instead. */ +static void +copy64( /* @out@ */ uint32_t * M, unsigned char const *in) +{ + int i; + + for (i = 0; i < 16; i++) + M[i] = (in[i * 4 + 3] << 24) | (in[i * 4 + 2] << 16) | + (in[i * 4 + 1] << 8) | (in[i * 4 + 0] << 0); +} + +static void +copy4( /* @out@ */ unsigned char *out, uint32_t const x) +{ + out[0] = x & 0xFF; + out[1] = (x >> 8) & 0xFF; + out[2] = (x >> 16) & 0xFF; + out[3] = (x >> 24) & 0xFF; +} + + + +/** + * Accumulate a block, making appropriate conversions for bigendian + * machines. + */ +static void +rs_mdfour_block_slow(rs_mdfour_t *md, void const *p) +{ + uint32_t M[16]; + + copy64(M, p); + rs_mdfour64(md, M); +} + + +static void +rs_mdfour_choose_packer(void) +{ + static const char foo[] = { 0xde, 0xad, 0xbe, 0xef}; + const uint32_t *p = (const uint32_t *) foo; + + if (sizeof(uint32_t) != 4) + rs_fatal("internal error: uint32_t is not really 32 bits!"); + if (sizeof(foo) != 4) + rs_fatal("internal error: something wierd about char arrays"); + + if (*p == 0xdeadbeef) { + rs_trace("big-endian machine"); + rs_mdfour_block = rs_mdfour_block_slow; + } else if (*p == 0xefbeadde) { + rs_trace("little-endian machine"); + rs_mdfour_block = rs_mdfour64; + } else { + rs_fatal("can't determine endianness from %#x", *p); + } +} + + +void +rs_mdfour_begin(rs_mdfour_t * md) +{ + if (!rs_mdfour_block) + rs_mdfour_choose_packer(); + + memset(md, 0, sizeof(*md)); + md->A = 0x67452301; + md->B = 0xefcdab89; + md->C = 0x98badcfe; + md->D = 0x10325476; + md->totalN = 0; +} + + +/** + * Handle special behaviour for processing the last block of a file + * when calculating its MD4 checksum. + * + * This must be called exactly once per file. + */ +static void +rs_mdfour_tail(rs_mdfour_t * m, unsigned char const *in, int n) +{ + unsigned char buf[128]; + uint32_t b; + + m->totalN += n; + + b = m->totalN * 8; + + memset(buf, 0, 128); + if (n) + memcpy(buf, in, n); + buf[n] = 0x80; + + if (n <= 55) { + copy4(buf + 56, b); + rs_mdfour_block(m, buf); + } else { + copy4(buf + 120, b); + rs_mdfour_block(m, buf); + rs_mdfour_block(m, buf + 64); + } +} + + +/** + * Feed some data into the MD4 accumulator. + * + * \param n Number of bytes fed in. + */ +void +rs_mdfour_update(rs_mdfour_t * md, void const *in_void, size_t n) +{ + unsigned char const *in = (unsigned char const *) in_void; + + if (n == 0) + return; + + if (md->tail_len) { + size_t tail_gap = 64 - md->tail_len; + + /* If there's any leftover data in the tail buffer, then first + * we have to make it up to a whole block and process it. */ + if (tail_gap > n) + tail_gap = n; + memcpy(&md->tail[md->tail_len], in, tail_gap); + md->tail_len += tail_gap; + in += tail_gap; + n -= tail_gap; + + if (md->tail_len != 64) + return; + + rs_mdfour_block(md, md->tail); + md->tail_len = 0; + md->totalN += 64; + } + + while (n >= 64) { + rs_mdfour_block(md, in); + in += 64; + n -= 64; + md->totalN += 64; + } + + if (n) { + memcpy(md->tail, in, n); + md->tail_len = n; + } +} + + +void +rs_mdfour_result(rs_mdfour_t * md, unsigned char *out) +{ + rs_mdfour_tail(md, md->tail, md->tail_len); + + copy4(out, md->A); + copy4(out + 4, md->B); + copy4(out + 8, md->C); + copy4(out + 12, md->D); +} + + +void +rs_mdfour(unsigned char *out, void const *in, size_t n) +{ + rs_mdfour_t md; + + rs_mdfour_begin(&md); + rs_mdfour_update(&md, in, n); + rs_mdfour_result(&md, out); +} diff --git a/rsync/mksum.c b/rsync/mksum.c new file mode 100644 index 0000000..d55bcfb --- a/dev/null +++ b/rsync/mksum.c @@ -0,0 +1,155 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell <tridge@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +/* + * mksum.c -- Generate file signatures. + * + * Generating checksums is pretty easy, since we can always just + * process whatever data is available. When a whole block has + * arrived, or we've reached the end of the file, we write the + * checksum out. + */ + +/* TODO: Perhaps force blocks to be a multiple of 64 bytes, so that we + * can be sure checksum generation will be more efficient. I guess it + * will be OK at the moment, though, because tails are only used if + * necessary. */ + +#include <config_rsync.h> + +#include <stdlib.h> +#include <stdio.h> +#include <assert.h> + +#include "rsync.h" +#include "stream.h" +#include "util.h" +#include "sumset.h" +#include "job.h" +#include "protocol.h" +#include "netint.h" +#include "trace.h" +#include "checksum.h" + + +/* Possible state functions for signature generation. */ +static rs_result rs_sig_s_header(rs_job_t *); +static rs_result rs_sig_s_generate(rs_job_t *); + + + +/** + * State of trying to send the signature header. + */ +static rs_result rs_sig_s_header(rs_job_t *job) +{ + rs_squirt_n4(job, RS_SIG_MAGIC); + rs_squirt_n4(job, job->block_len); + rs_squirt_n4(job, job->strong_sum_len); + rs_trace("sent header (magic %#x, block len = %d, strong sum len = %d)", + RS_SIG_MAGIC, (int) job->block_len, (int) job->strong_sum_len); + job->stats.block_len = job->block_len; + + job->statefn = rs_sig_s_generate; + return RS_RUNNING; +} + + +/** + * Generate the checksums for a block and write it out. Called when + * we already know we have enough data in memory at \p block. + */ +static rs_result +rs_sig_do_block(rs_job_t *job, const void *block, size_t len) +{ + unsigned int weak_sum; + rs_strong_sum_t strong_sum; + + weak_sum = rs_calc_weak_sum(block, len); + + rs_calc_strong_sum(block, len, &strong_sum); + + rs_squirt_n4(job, weak_sum); + rs_tube_write(job, strong_sum, job->strong_sum_len); + + if (rs_trace_enabled()) { + char strong_sum_hex[RS_MD4_LENGTH * 2 + 1]; + rs_hexify(strong_sum_hex, strong_sum, job->strong_sum_len); + rs_trace("sent weak sum 0x%08x and strong sum %s", weak_sum, + strong_sum_hex); + } + + job->stats.sig_blocks++; + + return RS_RUNNING; +} + + +/* + * State of reading a block and trying to generate its sum. + */ +static rs_result +rs_sig_s_generate(rs_job_t *job) +{ + rs_result result; + size_t len; + void *block; + + /* must get a whole block, otherwise try again */ + len = job->block_len; + result = rs_scoop_read(job, len, &block); + + /* unless we're near eof, in which case we'll accept + * whatever's in there */ + if ((result == RS_BLOCKED && rs_job_input_is_ending(job))) { + result = rs_scoop_read_rest(job, &len, &block); + } else if (result == RS_INPUT_ENDED) { + return RS_DONE; + } else if (result != RS_DONE) { + rs_trace("generate stopped: %s", rs_strerror(result)); + return result; + } + + rs_trace("got %d byte block", len); + + return rs_sig_do_block(job, block, len); +} + + +/** \brief Set up a new encoding job. + * + * \sa rs_sig_file() + */ +rs_job_t * rs_sig_begin(size_t new_block_len, size_t strong_sum_len) +{ + rs_job_t *job; + + job = rs_job_new("signature", rs_sig_s_header); + job->block_len = new_block_len; + + assert(strong_sum_len > 0 && strong_sum_len <= RS_MD4_LENGTH); + job->strong_sum_len = strong_sum_len; + + return job; +} diff --git a/rsync/msg.c b/rsync/msg.c new file mode 100644 index 0000000..aad2eb5 --- a/dev/null +++ b/rsync/msg.c @@ -0,0 +1,75 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + + /* + | Welcome to Arco AM/PM Mini-Market. We + | would like to advise our customers + | that any individual who offers to + | pump gas, wash windows or solicit + | products is not employed by or + | associated with this facility. We + | discourage any contact with these + | individuals and ask that you report + | any problems to uniformed personal + | inside. Thankyou for shopping at + | Arco, and have a nice day. + */ + +#include <config_rsync.h> + +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" + + +/** \brief Translate from rs_result to human-readable messages. */ +char const *rs_strerror(rs_result r) +{ + switch (r) { + case RS_DONE: + return "OK"; + case RS_RUNNING: + return "still running"; + case RS_BAD_MAGIC: + return "bad magic number at start of stream"; + case RS_BLOCKED: + return "blocked waiting for input or output buffers"; + case RS_INPUT_ENDED: + return "unexpected end of input"; + case RS_CORRUPT: + return "stream corrupt"; + case RS_UNIMPLEMENTED: + return "unimplemented case"; + case RS_MEM_ERROR: + return "out of memory"; + case RS_IO_ERROR: + return "IO error"; + case RS_SYNTAX_ERROR: + return "bad command line syntax"; + case RS_INTERNAL_ERROR: + return "library internal error"; + default: + return "unexplained problem"; + } +} diff --git a/rsync/netint.c b/rsync/netint.c new file mode 100644 index 0000000..71e3f9d --- a/dev/null +++ b/rsync/netint.c @@ -0,0 +1,185 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell <tridge@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* + | Ummm, well, OK. The network's the + | network, the computer's the + | computer. Sorry for the confusion. + | -- Sun Microsystems + */ + +/* + * Network-byte-order output to the tube. + * + * All the `suck' routines return a result code. The most common + * values are RS_DONE if they have enough data, or RS_BLOCKED if there + * is not enough input to proceed. + * + * All the netint operations are done in a fairly simpleminded way, + * since we don't want to rely on stdint types that may not be + * available on some platforms. + */ + +/* + * TODO: If we don't have <stdint.h> (or perhaps even if we do), + * determine endianness and integer size by hand and use that to do + * our own conversion routines. We possibly need this anyhow to do + * 64-bit integers, since there seems to be no ntohs() analog. + */ + +#include <config_rsync.h> + +#include <assert.h> +#include <sys/types.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> + +#include "rsync.h" + +#include "job.h" +#include "netint.h" +#include "trace.h" +#include "stream.h" + +#define RS_MAX_INT_BYTES 8 + + +/** + * \brief Write a single byte to a stream output. + */ +rs_result +rs_squirt_byte(rs_job_t *job, unsigned char d) +{ + rs_tube_write(job, &d, 1); + return RS_DONE; +} + + +/** + * \brief Write a variable-length integer to a stream. + * + * \param job Job of data. + * + * \param d Datum to write out. + * + * \param len Length of integer, in bytes. + */ +rs_result +rs_squirt_netint(rs_job_t *job, rs_long_t d, int len) +{ + unsigned char buf[RS_MAX_INT_BYTES]; + int i, j; + + if (len <= 0 || len > RS_MAX_INT_BYTES) { + rs_error("Illegal integer length %d", len); + return RS_INTERNAL_ERROR; + } + + /* Fill the output buffer with a bigendian representation of the + * number. */ + for (i = 0, j = len-1; i < len; i++, j--) { + buf[j] = d; /* truncated */ + d >>= 8; + } + + rs_tube_write(job, buf, len); + + return RS_DONE; +} + + + +rs_result +rs_squirt_n4(rs_job_t *job, int val) +{ + return rs_squirt_netint(job, val, 4); +} + + + +rs_result +rs_suck_netint(rs_job_t *job, rs_long_t *v, int len) +{ + unsigned char *buf; + int i; + rs_result result; + + if (len <= 0 || len > RS_MAX_INT_BYTES) { + rs_error("Illegal integer length %d", len); + return RS_INTERNAL_ERROR; + } + + if ((result = rs_scoop_read(job, len, (void **) &buf)) != RS_DONE) + return result; + + *v = 0; + + for (i = 0; i < len; i++) { + *v = *v<<8 | buf[i]; + } + + return RS_DONE; +} + + +rs_result +rs_suck_byte(rs_job_t *job, unsigned char *v) +{ + void *inb; + rs_result result; + + if ((result = rs_scoop_read(job, 1, &inb)) == RS_DONE) + *v = *((unsigned char *) inb); + + return result; +} + + +rs_result +rs_suck_n4(rs_job_t *job, int *v) +{ + rs_result result; + rs_long_t d; + + result = rs_suck_netint(job, &d, 4); + *v = d; + return result; +} + + +int rs_int_len(rs_long_t val) +{ + if (!(val & ~0xffL)) + return 1; + else if (!(val & ~0xffffL)) + return 2; + else if (!(val & ~0xffffffffL)) + return 4; + else if (!(val & ~0xffffffffffffffffL)) + return 8; + else { + rs_fatal("can't encode integer %.0f yet", (double) val); + } +} + diff --git a/rsync/netint.h b/rsync/netint.h new file mode 100644 index 0000000..ee02b94 --- a/dev/null +++ b/rsync/netint.h @@ -0,0 +1,32 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell <tridge@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +rs_result rs_squirt_byte(rs_job_t *, unsigned char d); +rs_result rs_squirt_netint(rs_job_t *, rs_long_t d, int len); +rs_result rs_squirt_n4(rs_job_t *, int val); + +rs_result rs_suck_netint(rs_job_t *, rs_long_t *v, int len); +rs_result rs_suck_byte(rs_job_t *, unsigned char *); +rs_result rs_suck_n4(rs_job_t *, int *); + +int rs_int_len(rs_long_t val); diff --git a/rsync/patch.c b/rsync/patch.c new file mode 100644 index 0000000..d2daa85 --- a/dev/null +++ b/rsync/patch.c @@ -0,0 +1,317 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + + /* + | This is Tranquility Base. + */ + + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" +#include "util.h" +#include "trace.h" +#include "protocol.h" +#include "netint.h" +#include "command.h" +#include "sumset.h" +#include "prototab.h" +#include "stream.h" +#include "job.h" + + + +static rs_result rs_patch_s_cmdbyte(rs_job_t *); +static rs_result rs_patch_s_params(rs_job_t *); +static rs_result rs_patch_s_run(rs_job_t *); +static rs_result rs_patch_s_literal(rs_job_t *); +static rs_result rs_patch_s_copy(rs_job_t *); +static rs_result rs_patch_s_copying(rs_job_t *); + + +/** + * State of trying to read the first byte of a command. Once we've + * taken that in, we can know how much data to read to get the + * arguments. + */ +static rs_result rs_patch_s_cmdbyte(rs_job_t *job) +{ + rs_result result; + + if ((result = rs_suck_byte(job, &job->op)) != RS_DONE) + return result; + + job->cmd = &rs_prototab[job->op]; + + rs_trace("got command byte 0x%02x (%s), len_1=%.0f", job->op, + rs_op_kind_name(job->cmd->kind), + (double) job->cmd->len_1); + + if (job->cmd->len_1) + job->statefn = rs_patch_s_params; + else { + job->param1 = job->cmd->immediate; + job->statefn = rs_patch_s_run; + } + + return RS_RUNNING; +} + + +/** + * Called after reading a command byte to pull in its parameters and + * then setup to execute the command. + */ +static rs_result rs_patch_s_params(rs_job_t *job) +{ + rs_result result; + int len = job->cmd->len_1 + job->cmd->len_2; + void *p; + + assert(len); + + result = rs_scoop_readahead(job, len, &p); + if (result != RS_DONE) + return result; + + /* we now must have LEN bytes buffered */ + result = rs_suck_netint(job, &job->param1, job->cmd->len_1); + /* shouldn't fail, since we already checked */ + assert(result == RS_DONE); + + if (job->cmd->len_2) { + result = rs_suck_netint(job, &job->param2, job->cmd->len_2); + assert(result == RS_DONE); + } + + job->statefn = rs_patch_s_run; + + return RS_RUNNING; +} + + + +/** + * Called when we've read in the whole command and we need to execute it. + */ +static rs_result rs_patch_s_run(rs_job_t *job) +{ + rs_trace("running command 0x%x, kind %d", job->op, job->cmd->kind); + + switch (job->cmd->kind) { + case RS_KIND_LITERAL: + job->statefn = rs_patch_s_literal; + return RS_RUNNING; + + case RS_KIND_END: + return RS_DONE; + /* so we exit here; trying to continue causes an error */ + + case RS_KIND_COPY: + job->statefn = rs_patch_s_copy; + return RS_RUNNING; + + default: + rs_error("bogus command 0x%02x", job->op); + return RS_CORRUPT; + } +} + + +/** + * Called when trying to copy through literal data. + */ +static rs_result rs_patch_s_literal(rs_job_t *job) +{ + rs_long_t len = job->param1; + + rs_trace("LITERAL(len=%.0f)", (double) len); + + job->stats.lit_cmds++; + job->stats.lit_bytes += len; + job->stats.lit_cmdbytes += 1 + job->cmd->len_1; + + rs_tube_copy(job, len); + + job->statefn = rs_patch_s_cmdbyte; + return RS_RUNNING; +} + + + +static rs_result rs_patch_s_copy(rs_job_t *job) +{ + rs_long_t where, len; + rs_stats_t *stats; + + where = job->param1; + len = job->param2; + + rs_trace("COPY(where=%.0f, len=%.0f)", (double) where, (double) len); + + if (len < 0) { + rs_log(RS_LOG_ERR, "invalid length=%.0f on COPY command", (double) len); + return RS_CORRUPT; + } + + if (where < 0) { + rs_log(RS_LOG_ERR, "invalid where=%.0f on COPY command", (double) where); + return RS_CORRUPT; + } + + job->basis_pos = where; + job->basis_len = len; + + stats = &job->stats; + + stats->copy_cmds++; + stats->copy_bytes += len; + stats->copy_cmdbytes += 1 + job->cmd->len_1 + job->cmd->len_2; + + job->statefn = rs_patch_s_copying; + return RS_RUNNING; +} + + +/** + * Called when we're executing a COPY command and waiting for all the + * data to be retrieved from the callback. + */ +static rs_result rs_patch_s_copying(rs_job_t *job) +{ + rs_result result; + size_t len; + void *buf, *ptr; + rs_buffers_t *buffs = job->stream; + + len = job->basis_len; + + /* copy only as much as will fit in the output buffer, so that we + * don't have to block or store the input. */ + if (len > buffs->avail_out) + len = buffs->avail_out; + + if (!len) + return RS_BLOCKED; + + rs_trace("copy %.0f bytes from basis at offset %.0f", + (double) len, (double) job->basis_pos); + + ptr = buf = rs_alloc(len, "basis buffer"); + + result = (job->copy_cb)(job->copy_arg, job->basis_pos, &len, &ptr); + if (result != RS_DONE) + return result; + else + rs_trace("copy callback returned %s", rs_strerror(result)); + + rs_trace("got %.0f bytes back from basis callback", (double) len); + + memcpy(buffs->next_out, ptr, len); + + buffs->next_out += len; + buffs->avail_out -= len; + + job->basis_pos += len; + job->basis_len -= len; + + free(buf); + + if (!job->basis_len) { + /* Done! */ + job->statefn = rs_patch_s_cmdbyte; + } + + return RS_RUNNING; +} + + +/** + * Called while we're trying to read the header of the patch. + */ +static rs_result rs_patch_s_header(rs_job_t *job) +{ + int v; + rs_result result; + + + if ((result = rs_suck_n4(job, &v)) != RS_DONE) + return result; + + if (v != RS_DELTA_MAGIC) { + rs_log(RS_LOG_ERR, + "got magic number %#x rather than expected value %#x", + v, RS_DELTA_MAGIC); + return RS_BAD_MAGIC; + } else + rs_trace("got patch magic %#x", v); + + + job->statefn = rs_patch_s_cmdbyte; + + return RS_RUNNING; +} + + + +/** + * \brief Apply a \ref gloss_delta to a \ref gloss_basis to recreate + * the new file. + * + * This gives you back a ::rs_job_t object, which can be cranked by + * calling rs_job_iter() and updating the stream pointers. When + * finished, call rs_job_finish() to dispose of it. + * + * \param stream Contains pointers to input and output buffers, to be + * adjusted by caller on each iteration. + * + * \param copy_cb Callback used to retrieve content from the basis + * file. + * + * \param copy_arg Opaque environment pointer passed through to the + * callback. + * + * \todo As output is produced, accumulate the MD4 checksum of the + * output. Then if we find a CHECKSUM command we can check it's + * contents against the output. + * + * \todo Implement COPY commands. + * + * \sa rs_patch_file() + */ +rs_job_t * +rs_patch_begin(rs_copy_cb *copy_cb, void *copy_arg) +{ + rs_job_t *job = rs_job_new("patch", rs_patch_s_header); + + job->copy_cb = copy_cb; + job->copy_arg = copy_arg; + + rs_mdfour_begin(&job->output_md4); + + return job; +} diff --git a/rsync/protocol.h b/rsync/protocol.h new file mode 100644 index 0000000..bee63f1 --- a/dev/null +++ b/rsync/protocol.h @@ -0,0 +1,45 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. */ + + +/* + * TODO: Have a way to copy from the old signature into the new + * one. This will be useful for the case where the files are in fact + * identical, which will be significantly common. + */ + + + /* + | "The IETF already has more than enough + | RFCs that codify the obvious, make + | stupidity illegal, support truth, + | justice, and the IETF way, and generally + | demonstrate the author is a brilliant and + | valuable Contributor to The Standards + | Process." + | -- Vernon Schryver + */ + + + +#define RS_DELTA_MAGIC 0x72730236 /* r s \2 6 */ +#define RS_SIG_MAGIC 0x72730136 /* r s \1 6 */ diff --git a/rsync/prototab.c b/rsync/prototab.c new file mode 100644 index 0000000..bce7d42 --- a/dev/null +++ b/rsync/prototab.c @@ -0,0 +1,277 @@ + +/* AUTOGENERATED BY ./mkprototab.pl, DO NOT EDIT */ + +#include <config_rsync.h> + +#include <stdlib.h> +#include <stdio.h> + +#include <rsync.h> +#include <protocol.h> +#include <command.h> +#include <prototab.h> + +/* This file defines an array mapping command IDs to the operation kind, + * implied literal value, and length of the first and second parameters. + * The implied value is only used + * if the first parameter length is zero. */ + +const struct rs_prototab_ent rs_prototab[] = { + {RS_KIND_END , 0, 0, 0 } /* RS_OP_END = 0 */, + {RS_KIND_LITERAL , 1, 0, 0 } /* RS_OP_LITERAL_1 = 0x1 */, + {RS_KIND_LITERAL , 2, 0, 0 } /* RS_OP_LITERAL_2 = 0x2 */, + {RS_KIND_LITERAL , 3, 0, 0 } /* RS_OP_LITERAL_3 = 0x3 */, + {RS_KIND_LITERAL , 4, 0, 0 } /* RS_OP_LITERAL_4 = 0x4 */, + {RS_KIND_LITERAL , 5, 0, 0 } /* RS_OP_LITERAL_5 = 0x5 */, + {RS_KIND_LITERAL , 6, 0, 0 } /* RS_OP_LITERAL_6 = 0x6 */, + {RS_KIND_LITERAL , 7, 0, 0 } /* RS_OP_LITERAL_7 = 0x7 */, + {RS_KIND_LITERAL , 8, 0, 0 } /* RS_OP_LITERAL_8 = 0x8 */, + {RS_KIND_LITERAL , 9, 0, 0 } /* RS_OP_LITERAL_9 = 0x9 */, + {RS_KIND_LITERAL , 10, 0, 0 } /* RS_OP_LITERAL_10 = 0xa */, + {RS_KIND_LITERAL , 11, 0, 0 } /* RS_OP_LITERAL_11 = 0xb */, + {RS_KIND_LITERAL , 12, 0, 0 } /* RS_OP_LITERAL_12 = 0xc */, + {RS_KIND_LITERAL , 13, 0, 0 } /* RS_OP_LITERAL_13 = 0xd */, + {RS_KIND_LITERAL , 14, 0, 0 } /* RS_OP_LITERAL_14 = 0xe */, + {RS_KIND_LITERAL , 15, 0, 0 } /* RS_OP_LITERAL_15 = 0xf */, + {RS_KIND_LITERAL , 16, 0, 0 } /* RS_OP_LITERAL_16 = 0x10 */, + {RS_KIND_LITERAL , 17, 0, 0 } /* RS_OP_LITERAL_17 = 0x11 */, + {RS_KIND_LITERAL , 18, 0, 0 } /* RS_OP_LITERAL_18 = 0x12 */, + {RS_KIND_LITERAL , 19, 0, 0 } /* RS_OP_LITERAL_19 = 0x13 */, + {RS_KIND_LITERAL , 20, 0, 0 } /* RS_OP_LITERAL_20 = 0x14 */, + {RS_KIND_LITERAL , 21, 0, 0 } /* RS_OP_LITERAL_21 = 0x15 */, + {RS_KIND_LITERAL , 22, 0, 0 } /* RS_OP_LITERAL_22 = 0x16 */, + {RS_KIND_LITERAL , 23, 0, 0 } /* RS_OP_LITERAL_23 = 0x17 */, + {RS_KIND_LITERAL , 24, 0, 0 } /* RS_OP_LITERAL_24 = 0x18 */, + {RS_KIND_LITERAL , 25, 0, 0 } /* RS_OP_LITERAL_25 = 0x19 */, + {RS_KIND_LITERAL , 26, 0, 0 } /* RS_OP_LITERAL_26 = 0x1a */, + {RS_KIND_LITERAL , 27, 0, 0 } /* RS_OP_LITERAL_27 = 0x1b */, + {RS_KIND_LITERAL , 28, 0, 0 } /* RS_OP_LITERAL_28 = 0x1c */, + {RS_KIND_LITERAL , 29, 0, 0 } /* RS_OP_LITERAL_29 = 0x1d */, + {RS_KIND_LITERAL , 30, 0, 0 } /* RS_OP_LITERAL_30 = 0x1e */, + {RS_KIND_LITERAL , 31, 0, 0 } /* RS_OP_LITERAL_31 = 0x1f */, + {RS_KIND_LITERAL , 32, 0, 0 } /* RS_OP_LITERAL_32 = 0x20 */, + {RS_KIND_LITERAL , 33, 0, 0 } /* RS_OP_LITERAL_33 = 0x21 */, + {RS_KIND_LITERAL , 34, 0, 0 } /* RS_OP_LITERAL_34 = 0x22 */, + {RS_KIND_LITERAL , 35, 0, 0 } /* RS_OP_LITERAL_35 = 0x23 */, + {RS_KIND_LITERAL , 36, 0, 0 } /* RS_OP_LITERAL_36 = 0x24 */, + {RS_KIND_LITERAL , 37, 0, 0 } /* RS_OP_LITERAL_37 = 0x25 */, + {RS_KIND_LITERAL , 38, 0, 0 } /* RS_OP_LITERAL_38 = 0x26 */, + {RS_KIND_LITERAL , 39, 0, 0 } /* RS_OP_LITERAL_39 = 0x27 */, + {RS_KIND_LITERAL , 40, 0, 0 } /* RS_OP_LITERAL_40 = 0x28 */, + {RS_KIND_LITERAL , 41, 0, 0 } /* RS_OP_LITERAL_41 = 0x29 */, + {RS_KIND_LITERAL , 42, 0, 0 } /* RS_OP_LITERAL_42 = 0x2a */, + {RS_KIND_LITERAL , 43, 0, 0 } /* RS_OP_LITERAL_43 = 0x2b */, + {RS_KIND_LITERAL , 44, 0, 0 } /* RS_OP_LITERAL_44 = 0x2c */, + {RS_KIND_LITERAL , 45, 0, 0 } /* RS_OP_LITERAL_45 = 0x2d */, + {RS_KIND_LITERAL , 46, 0, 0 } /* RS_OP_LITERAL_46 = 0x2e */, + {RS_KIND_LITERAL , 47, 0, 0 } /* RS_OP_LITERAL_47 = 0x2f */, + {RS_KIND_LITERAL , 48, 0, 0 } /* RS_OP_LITERAL_48 = 0x30 */, + {RS_KIND_LITERAL , 49, 0, 0 } /* RS_OP_LITERAL_49 = 0x31 */, + {RS_KIND_LITERAL , 50, 0, 0 } /* RS_OP_LITERAL_50 = 0x32 */, + {RS_KIND_LITERAL , 51, 0, 0 } /* RS_OP_LITERAL_51 = 0x33 */, + {RS_KIND_LITERAL , 52, 0, 0 } /* RS_OP_LITERAL_52 = 0x34 */, + {RS_KIND_LITERAL , 53, 0, 0 } /* RS_OP_LITERAL_53 = 0x35 */, + {RS_KIND_LITERAL , 54, 0, 0 } /* RS_OP_LITERAL_54 = 0x36 */, + {RS_KIND_LITERAL , 55, 0, 0 } /* RS_OP_LITERAL_55 = 0x37 */, + {RS_KIND_LITERAL , 56, 0, 0 } /* RS_OP_LITERAL_56 = 0x38 */, + {RS_KIND_LITERAL , 57, 0, 0 } /* RS_OP_LITERAL_57 = 0x39 */, + {RS_KIND_LITERAL , 58, 0, 0 } /* RS_OP_LITERAL_58 = 0x3a */, + {RS_KIND_LITERAL , 59, 0, 0 } /* RS_OP_LITERAL_59 = 0x3b */, + {RS_KIND_LITERAL , 60, 0, 0 } /* RS_OP_LITERAL_60 = 0x3c */, + {RS_KIND_LITERAL , 61, 0, 0 } /* RS_OP_LITERAL_61 = 0x3d */, + {RS_KIND_LITERAL , 62, 0, 0 } /* RS_OP_LITERAL_62 = 0x3e */, + {RS_KIND_LITERAL , 63, 0, 0 } /* RS_OP_LITERAL_63 = 0x3f */, + {RS_KIND_LITERAL , 64, 0, 0 } /* RS_OP_LITERAL_64 = 0x40 */, + {RS_KIND_LITERAL , 0, 1, 0 } /* RS_OP_LITERAL_N1 = 0x41 */, + {RS_KIND_LITERAL , 0, 2, 0 } /* RS_OP_LITERAL_N2 = 0x42 */, + {RS_KIND_LITERAL , 0, 4, 0 } /* RS_OP_LITERAL_N4 = 0x43 */, + {RS_KIND_LITERAL , 0, 8, 0 } /* RS_OP_LITERAL_N8 = 0x44 */, + {RS_KIND_COPY , 0, 1, 1 } /* RS_OP_COPY_N1_N1 = 0x45 */, + {RS_KIND_COPY , 0, 1, 2 } /* RS_OP_COPY_N1_N2 = 0x46 */, + {RS_KIND_COPY , 0, 1, 4 } /* RS_OP_COPY_N1_N4 = 0x47 */, + {RS_KIND_COPY , 0, 1, 8 } /* RS_OP_COPY_N1_N8 = 0x48 */, + {RS_KIND_COPY , 0, 2, 1 } /* RS_OP_COPY_N2_N1 = 0x49 */, + {RS_KIND_COPY , 0, 2, 2 } /* RS_OP_COPY_N2_N2 = 0x4a */, + {RS_KIND_COPY , 0, 2, 4 } /* RS_OP_COPY_N2_N4 = 0x4b */, + {RS_KIND_COPY , 0, 2, 8 } /* RS_OP_COPY_N2_N8 = 0x4c */, + {RS_KIND_COPY , 0, 4, 1 } /* RS_OP_COPY_N4_N1 = 0x4d */, + {RS_KIND_COPY , 0, 4, 2 } /* RS_OP_COPY_N4_N2 = 0x4e */, + {RS_KIND_COPY , 0, 4, 4 } /* RS_OP_COPY_N4_N4 = 0x4f */, + {RS_KIND_COPY , 0, 4, 8 } /* RS_OP_COPY_N4_N8 = 0x50 */, + {RS_KIND_COPY , 0, 8, 1 } /* RS_OP_COPY_N8_N1 = 0x51 */, + {RS_KIND_COPY , 0, 8, 2 } /* RS_OP_COPY_N8_N2 = 0x52 */, + {RS_KIND_COPY , 0, 8, 4 } /* RS_OP_COPY_N8_N4 = 0x53 */, + {RS_KIND_COPY , 0, 8, 8 } /* RS_OP_COPY_N8_N8 = 0x54 */, + {RS_KIND_RESERVED , 85, 0, 0 } /* RS_OP_RESERVED_85 = 0x55 */, + {RS_KIND_RESERVED , 86, 0, 0 } /* RS_OP_RESERVED_86 = 0x56 */, + {RS_KIND_RESERVED , 87, 0, 0 } /* RS_OP_RESERVED_87 = 0x57 */, + {RS_KIND_RESERVED , 88, 0, 0 } /* RS_OP_RESERVED_88 = 0x58 */, + {RS_KIND_RESERVED , 89, 0, 0 } /* RS_OP_RESERVED_89 = 0x59 */, + {RS_KIND_RESERVED , 90, 0, 0 } /* RS_OP_RESERVED_90 = 0x5a */, + {RS_KIND_RESERVED , 91, 0, 0 } /* RS_OP_RESERVED_91 = 0x5b */, + {RS_KIND_RESERVED , 92, 0, 0 } /* RS_OP_RESERVED_92 = 0x5c */, + {RS_KIND_RESERVED , 93, 0, 0 } /* RS_OP_RESERVED_93 = 0x5d */, + {RS_KIND_RESERVED , 94, 0, 0 } /* RS_OP_RESERVED_94 = 0x5e */, + {RS_KIND_RESERVED , 95, 0, 0 } /* RS_OP_RESERVED_95 = 0x5f */, + {RS_KIND_RESERVED , 96, 0, 0 } /* RS_OP_RESERVED_96 = 0x60 */, + {RS_KIND_RESERVED , 97, 0, 0 } /* RS_OP_RESERVED_97 = 0x61 */, + {RS_KIND_RESERVED , 98, 0, 0 } /* RS_OP_RESERVED_98 = 0x62 */, + {RS_KIND_RESERVED , 99, 0, 0 } /* RS_OP_RESERVED_99 = 0x63 */, + {RS_KIND_RESERVED , 100, 0, 0 } /* RS_OP_RESERVED_100 = 0x64 */, + {RS_KIND_RESERVED , 101, 0, 0 } /* RS_OP_RESERVED_101 = 0x65 */, + {RS_KIND_RESERVED , 102, 0, 0 } /* RS_OP_RESERVED_102 = 0x66 */, + {RS_KIND_RESERVED , 103, 0, 0 } /* RS_OP_RESERVED_103 = 0x67 */, + {RS_KIND_RESERVED , 104, 0, 0 } /* RS_OP_RESERVED_104 = 0x68 */, + {RS_KIND_RESERVED , 105, 0, 0 } /* RS_OP_RESERVED_105 = 0x69 */, + {RS_KIND_RESERVED , 106, 0, 0 } /* RS_OP_RESERVED_106 = 0x6a */, + {RS_KIND_RESERVED , 107, 0, 0 } /* RS_OP_RESERVED_107 = 0x6b */, + {RS_KIND_RESERVED , 108, 0, 0 } /* RS_OP_RESERVED_108 = 0x6c */, + {RS_KIND_RESERVED , 109, 0, 0 } /* RS_OP_RESERVED_109 = 0x6d */, + {RS_KIND_RESERVED , 110, 0, 0 } /* RS_OP_RESERVED_110 = 0x6e */, + {RS_KIND_RESERVED , 111, 0, 0 } /* RS_OP_RESERVED_111 = 0x6f */, + {RS_KIND_RESERVED , 112, 0, 0 } /* RS_OP_RESERVED_112 = 0x70 */, + {RS_KIND_RESERVED , 113, 0, 0 } /* RS_OP_RESERVED_113 = 0x71 */, + {RS_KIND_RESERVED , 114, 0, 0 } /* RS_OP_RESERVED_114 = 0x72 */, + {RS_KIND_RESERVED , 115, 0, 0 } /* RS_OP_RESERVED_115 = 0x73 */, + {RS_KIND_RESERVED , 116, 0, 0 } /* RS_OP_RESERVED_116 = 0x74 */, + {RS_KIND_RESERVED , 117, 0, 0 } /* RS_OP_RESERVED_117 = 0x75 */, + {RS_KIND_RESERVED , 118, 0, 0 } /* RS_OP_RESERVED_118 = 0x76 */, + {RS_KIND_RESERVED , 119, 0, 0 } /* RS_OP_RESERVED_119 = 0x77 */, + {RS_KIND_RESERVED , 120, 0, 0 } /* RS_OP_RESERVED_120 = 0x78 */, + {RS_KIND_RESERVED , 121, 0, 0 } /* RS_OP_RESERVED_121 = 0x79 */, + {RS_KIND_RESERVED , 122, 0, 0 } /* RS_OP_RESERVED_122 = 0x7a */, + {RS_KIND_RESERVED , 123, 0, 0 } /* RS_OP_RESERVED_123 = 0x7b */, + {RS_KIND_RESERVED , 124, 0, 0 } /* RS_OP_RESERVED_124 = 0x7c */, + {RS_KIND_RESERVED , 125, 0, 0 } /* RS_OP_RESERVED_125 = 0x7d */, + {RS_KIND_RESERVED , 126, 0, 0 } /* RS_OP_RESERVED_126 = 0x7e */, + {RS_KIND_RESERVED , 127, 0, 0 } /* RS_OP_RESERVED_127 = 0x7f */, + {RS_KIND_RESERVED , 128, 0, 0 } /* RS_OP_RESERVED_128 = 0x80 */, + {RS_KIND_RESERVED , 129, 0, 0 } /* RS_OP_RESERVED_129 = 0x81 */, + {RS_KIND_RESERVED , 130, 0, 0 } /* RS_OP_RESERVED_130 = 0x82 */, + {RS_KIND_RESERVED , 131, 0, 0 } /* RS_OP_RESERVED_131 = 0x83 */, + {RS_KIND_RESERVED , 132, 0, 0 } /* RS_OP_RESERVED_132 = 0x84 */, + {RS_KIND_RESERVED , 133, 0, 0 } /* RS_OP_RESERVED_133 = 0x85 */, + {RS_KIND_RESERVED , 134, 0, 0 } /* RS_OP_RESERVED_134 = 0x86 */, + {RS_KIND_RESERVED , 135, 0, 0 } /* RS_OP_RESERVED_135 = 0x87 */, + {RS_KIND_RESERVED , 136, 0, 0 } /* RS_OP_RESERVED_136 = 0x88 */, + {RS_KIND_RESERVED , 137, 0, 0 } /* RS_OP_RESERVED_137 = 0x89 */, + {RS_KIND_RESERVED , 138, 0, 0 } /* RS_OP_RESERVED_138 = 0x8a */, + {RS_KIND_RESERVED , 139, 0, 0 } /* RS_OP_RESERVED_139 = 0x8b */, + {RS_KIND_RESERVED , 140, 0, 0 } /* RS_OP_RESERVED_140 = 0x8c */, + {RS_KIND_RESERVED , 141, 0, 0 } /* RS_OP_RESERVED_141 = 0x8d */, + {RS_KIND_RESERVED , 142, 0, 0 } /* RS_OP_RESERVED_142 = 0x8e */, + {RS_KIND_RESERVED , 143, 0, 0 } /* RS_OP_RESERVED_143 = 0x8f */, + {RS_KIND_RESERVED , 144, 0, 0 } /* RS_OP_RESERVED_144 = 0x90 */, + {RS_KIND_RESERVED , 145, 0, 0 } /* RS_OP_RESERVED_145 = 0x91 */, + {RS_KIND_RESERVED , 146, 0, 0 } /* RS_OP_RESERVED_146 = 0x92 */, + {RS_KIND_RESERVED , 147, 0, 0 } /* RS_OP_RESERVED_147 = 0x93 */, + {RS_KIND_RESERVED , 148, 0, 0 } /* RS_OP_RESERVED_148 = 0x94 */, + {RS_KIND_RESERVED , 149, 0, 0 } /* RS_OP_RESERVED_149 = 0x95 */, + {RS_KIND_RESERVED , 150, 0, 0 } /* RS_OP_RESERVED_150 = 0x96 */, + {RS_KIND_RESERVED , 151, 0, 0 } /* RS_OP_RESERVED_151 = 0x97 */, + {RS_KIND_RESERVED , 152, 0, 0 } /* RS_OP_RESERVED_152 = 0x98 */, + {RS_KIND_RESERVED , 153, 0, 0 } /* RS_OP_RESERVED_153 = 0x99 */, + {RS_KIND_RESERVED , 154, 0, 0 } /* RS_OP_RESERVED_154 = 0x9a */, + {RS_KIND_RESERVED , 155, 0, 0 } /* RS_OP_RESERVED_155 = 0x9b */, + {RS_KIND_RESERVED , 156, 0, 0 } /* RS_OP_RESERVED_156 = 0x9c */, + {RS_KIND_RESERVED , 157, 0, 0 } /* RS_OP_RESERVED_157 = 0x9d */, + {RS_KIND_RESERVED , 158, 0, 0 } /* RS_OP_RESERVED_158 = 0x9e */, + {RS_KIND_RESERVED , 159, 0, 0 } /* RS_OP_RESERVED_159 = 0x9f */, + {RS_KIND_RESERVED , 160, 0, 0 } /* RS_OP_RESERVED_160 = 0xa0 */, + {RS_KIND_RESERVED , 161, 0, 0 } /* RS_OP_RESERVED_161 = 0xa1 */, + {RS_KIND_RESERVED , 162, 0, 0 } /* RS_OP_RESERVED_162 = 0xa2 */, + {RS_KIND_RESERVED , 163, 0, 0 } /* RS_OP_RESERVED_163 = 0xa3 */, + {RS_KIND_RESERVED , 164, 0, 0 } /* RS_OP_RESERVED_164 = 0xa4 */, + {RS_KIND_RESERVED , 165, 0, 0 } /* RS_OP_RESERVED_165 = 0xa5 */, + {RS_KIND_RESERVED , 166, 0, 0 } /* RS_OP_RESERVED_166 = 0xa6 */, + {RS_KIND_RESERVED , 167, 0, 0 } /* RS_OP_RESERVED_167 = 0xa7 */, + {RS_KIND_RESERVED , 168, 0, 0 } /* RS_OP_RESERVED_168 = 0xa8 */, + {RS_KIND_RESERVED , 169, 0, 0 } /* RS_OP_RESERVED_169 = 0xa9 */, + {RS_KIND_RESERVED , 170, 0, 0 } /* RS_OP_RESERVED_170 = 0xaa */, + {RS_KIND_RESERVED , 171, 0, 0 } /* RS_OP_RESERVED_171 = 0xab */, + {RS_KIND_RESERVED , 172, 0, 0 } /* RS_OP_RESERVED_172 = 0xac */, + {RS_KIND_RESERVED , 173, 0, 0 } /* RS_OP_RESERVED_173 = 0xad */, + {RS_KIND_RESERVED , 174, 0, 0 } /* RS_OP_RESERVED_174 = 0xae */, + {RS_KIND_RESERVED , 175, 0, 0 } /* RS_OP_RESERVED_175 = 0xaf */, + {RS_KIND_RESERVED , 176, 0, 0 } /* RS_OP_RESERVED_176 = 0xb0 */, + {RS_KIND_RESERVED , 177, 0, 0 } /* RS_OP_RESERVED_177 = 0xb1 */, + {RS_KIND_RESERVED , 178, 0, 0 } /* RS_OP_RESERVED_178 = 0xb2 */, + {RS_KIND_RESERVED , 179, 0, 0 } /* RS_OP_RESERVED_179 = 0xb3 */, + {RS_KIND_RESERVED , 180, 0, 0 } /* RS_OP_RESERVED_180 = 0xb4 */, + {RS_KIND_RESERVED , 181, 0, 0 } /* RS_OP_RESERVED_181 = 0xb5 */, + {RS_KIND_RESERVED , 182, 0, 0 } /* RS_OP_RESERVED_182 = 0xb6 */, + {RS_KIND_RESERVED , 183, 0, 0 } /* RS_OP_RESERVED_183 = 0xb7 */, + {RS_KIND_RESERVED , 184, 0, 0 } /* RS_OP_RESERVED_184 = 0xb8 */, + {RS_KIND_RESERVED , 185, 0, 0 } /* RS_OP_RESERVED_185 = 0xb9 */, + {RS_KIND_RESERVED , 186, 0, 0 } /* RS_OP_RESERVED_186 = 0xba */, + {RS_KIND_RESERVED , 187, 0, 0 } /* RS_OP_RESERVED_187 = 0xbb */, + {RS_KIND_RESERVED , 188, 0, 0 } /* RS_OP_RESERVED_188 = 0xbc */, + {RS_KIND_RESERVED , 189, 0, 0 } /* RS_OP_RESERVED_189 = 0xbd */, + {RS_KIND_RESERVED , 190, 0, 0 } /* RS_OP_RESERVED_190 = 0xbe */, + {RS_KIND_RESERVED , 191, 0, 0 } /* RS_OP_RESERVED_191 = 0xbf */, + {RS_KIND_RESERVED , 192, 0, 0 } /* RS_OP_RESERVED_192 = 0xc0 */, + {RS_KIND_RESERVED , 193, 0, 0 } /* RS_OP_RESERVED_193 = 0xc1 */, + {RS_KIND_RESERVED , 194, 0, 0 } /* RS_OP_RESERVED_194 = 0xc2 */, + {RS_KIND_RESERVED , 195, 0, 0 } /* RS_OP_RESERVED_195 = 0xc3 */, + {RS_KIND_RESERVED , 196, 0, 0 } /* RS_OP_RESERVED_196 = 0xc4 */, + {RS_KIND_RESERVED , 197, 0, 0 } /* RS_OP_RESERVED_197 = 0xc5 */, + {RS_KIND_RESERVED , 198, 0, 0 } /* RS_OP_RESERVED_198 = 0xc6 */, + {RS_KIND_RESERVED , 199, 0, 0 } /* RS_OP_RESERVED_199 = 0xc7 */, + {RS_KIND_RESERVED , 200, 0, 0 } /* RS_OP_RESERVED_200 = 0xc8 */, + {RS_KIND_RESERVED , 201, 0, 0 } /* RS_OP_RESERVED_201 = 0xc9 */, + {RS_KIND_RESERVED , 202, 0, 0 } /* RS_OP_RESERVED_202 = 0xca */, + {RS_KIND_RESERVED , 203, 0, 0 } /* RS_OP_RESERVED_203 = 0xcb */, + {RS_KIND_RESERVED , 204, 0, 0 } /* RS_OP_RESERVED_204 = 0xcc */, + {RS_KIND_RESERVED , 205, 0, 0 } /* RS_OP_RESERVED_205 = 0xcd */, + {RS_KIND_RESERVED , 206, 0, 0 } /* RS_OP_RESERVED_206 = 0xce */, + {RS_KIND_RESERVED , 207, 0, 0 } /* RS_OP_RESERVED_207 = 0xcf */, + {RS_KIND_RESERVED , 208, 0, 0 } /* RS_OP_RESERVED_208 = 0xd0 */, + {RS_KIND_RESERVED , 209, 0, 0 } /* RS_OP_RESERVED_209 = 0xd1 */, + {RS_KIND_RESERVED , 210, 0, 0 } /* RS_OP_RESERVED_210 = 0xd2 */, + {RS_KIND_RESERVED , 211, 0, 0 } /* RS_OP_RESERVED_211 = 0xd3 */, + {RS_KIND_RESERVED , 212, 0, 0 } /* RS_OP_RESERVED_212 = 0xd4 */, + {RS_KIND_RESERVED , 213, 0, 0 } /* RS_OP_RESERVED_213 = 0xd5 */, + {RS_KIND_RESERVED , 214, 0, 0 } /* RS_OP_RESERVED_214 = 0xd6 */, + {RS_KIND_RESERVED , 215, 0, 0 } /* RS_OP_RESERVED_215 = 0xd7 */, + {RS_KIND_RESERVED , 216, 0, 0 } /* RS_OP_RESERVED_216 = 0xd8 */, + {RS_KIND_RESERVED , 217, 0, 0 } /* RS_OP_RESERVED_217 = 0xd9 */, + {RS_KIND_RESERVED , 218, 0, 0 } /* RS_OP_RESERVED_218 = 0xda */, + {RS_KIND_RESERVED , 219, 0, 0 } /* RS_OP_RESERVED_219 = 0xdb */, + {RS_KIND_RESERVED , 220, 0, 0 } /* RS_OP_RESERVED_220 = 0xdc */, + {RS_KIND_RESERVED , 221, 0, 0 } /* RS_OP_RESERVED_221 = 0xdd */, + {RS_KIND_RESERVED , 222, 0, 0 } /* RS_OP_RESERVED_222 = 0xde */, + {RS_KIND_RESERVED , 223, 0, 0 } /* RS_OP_RESERVED_223 = 0xdf */, + {RS_KIND_RESERVED , 224, 0, 0 } /* RS_OP_RESERVED_224 = 0xe0 */, + {RS_KIND_RESERVED , 225, 0, 0 } /* RS_OP_RESERVED_225 = 0xe1 */, + {RS_KIND_RESERVED , 226, 0, 0 } /* RS_OP_RESERVED_226 = 0xe2 */, + {RS_KIND_RESERVED , 227, 0, 0 } /* RS_OP_RESERVED_227 = 0xe3 */, + {RS_KIND_RESERVED , 228, 0, 0 } /* RS_OP_RESERVED_228 = 0xe4 */, + {RS_KIND_RESERVED , 229, 0, 0 } /* RS_OP_RESERVED_229 = 0xe5 */, + {RS_KIND_RESERVED , 230, 0, 0 } /* RS_OP_RESERVED_230 = 0xe6 */, + {RS_KIND_RESERVED , 231, 0, 0 } /* RS_OP_RESERVED_231 = 0xe7 */, + {RS_KIND_RESERVED , 232, 0, 0 } /* RS_OP_RESERVED_232 = 0xe8 */, + {RS_KIND_RESERVED , 233, 0, 0 } /* RS_OP_RESERVED_233 = 0xe9 */, + {RS_KIND_RESERVED , 234, 0, 0 } /* RS_OP_RESERVED_234 = 0xea */, + {RS_KIND_RESERVED , 235, 0, 0 } /* RS_OP_RESERVED_235 = 0xeb */, + {RS_KIND_RESERVED , 236, 0, 0 } /* RS_OP_RESERVED_236 = 0xec */, + {RS_KIND_RESERVED , 237, 0, 0 } /* RS_OP_RESERVED_237 = 0xed */, + {RS_KIND_RESERVED , 238, 0, 0 } /* RS_OP_RESERVED_238 = 0xee */, + {RS_KIND_RESERVED , 239, 0, 0 } /* RS_OP_RESERVED_239 = 0xef */, + {RS_KIND_RESERVED , 240, 0, 0 } /* RS_OP_RESERVED_240 = 0xf0 */, + {RS_KIND_RESERVED , 241, 0, 0 } /* RS_OP_RESERVED_241 = 0xf1 */, + {RS_KIND_RESERVED , 242, 0, 0 } /* RS_OP_RESERVED_242 = 0xf2 */, + {RS_KIND_RESERVED , 243, 0, 0 } /* RS_OP_RESERVED_243 = 0xf3 */, + {RS_KIND_RESERVED , 244, 0, 0 } /* RS_OP_RESERVED_244 = 0xf4 */, + {RS_KIND_RESERVED , 245, 0, 0 } /* RS_OP_RESERVED_245 = 0xf5 */, + {RS_KIND_RESERVED , 246, 0, 0 } /* RS_OP_RESERVED_246 = 0xf6 */, + {RS_KIND_RESERVED , 247, 0, 0 } /* RS_OP_RESERVED_247 = 0xf7 */, + {RS_KIND_RESERVED , 248, 0, 0 } /* RS_OP_RESERVED_248 = 0xf8 */, + {RS_KIND_RESERVED , 249, 0, 0 } /* RS_OP_RESERVED_249 = 0xf9 */, + {RS_KIND_RESERVED , 250, 0, 0 } /* RS_OP_RESERVED_250 = 0xfa */, + {RS_KIND_RESERVED , 251, 0, 0 } /* RS_OP_RESERVED_251 = 0xfb */, + {RS_KIND_RESERVED , 252, 0, 0 } /* RS_OP_RESERVED_252 = 0xfc */, + {RS_KIND_RESERVED , 253, 0, 0 } /* RS_OP_RESERVED_253 = 0xfd */, + {RS_KIND_RESERVED , 254, 0, 0 } /* RS_OP_RESERVED_254 = 0xfe */, + {RS_KIND_RESERVED , 255, 0, 0 } /* RS_OP_RESERVED_255 = 0xff */ +}; +/* END OF AUTOGENERATED CODE */ diff --git a/rsync/prototab.h b/rsync/prototab.h new file mode 100644 index 0000000..3a9d771 --- a/dev/null +++ b/rsync/prototab.h @@ -0,0 +1,270 @@ + +/* AUTOGENERATED BY ./mkprototab.pl, DO NOT EDIT */ + +typedef struct rs_prototab_ent { + enum rs_op_kind kind; + int immediate; + size_t len_1, len_2; +} rs_prototab_ent_t; + +extern const rs_prototab_ent_t rs_prototab[]; + +enum { + RS_OP_END = 0, + RS_OP_LITERAL_1 = 0x1, + RS_OP_LITERAL_2 = 0x2, + RS_OP_LITERAL_3 = 0x3, + RS_OP_LITERAL_4 = 0x4, + RS_OP_LITERAL_5 = 0x5, + RS_OP_LITERAL_6 = 0x6, + RS_OP_LITERAL_7 = 0x7, + RS_OP_LITERAL_8 = 0x8, + RS_OP_LITERAL_9 = 0x9, + RS_OP_LITERAL_10 = 0xa, + RS_OP_LITERAL_11 = 0xb, + RS_OP_LITERAL_12 = 0xc, + RS_OP_LITERAL_13 = 0xd, + RS_OP_LITERAL_14 = 0xe, + RS_OP_LITERAL_15 = 0xf, + RS_OP_LITERAL_16 = 0x10, + RS_OP_LITERAL_17 = 0x11, + RS_OP_LITERAL_18 = 0x12, + RS_OP_LITERAL_19 = 0x13, + RS_OP_LITERAL_20 = 0x14, + RS_OP_LITERAL_21 = 0x15, + RS_OP_LITERAL_22 = 0x16, + RS_OP_LITERAL_23 = 0x17, + RS_OP_LITERAL_24 = 0x18, + RS_OP_LITERAL_25 = 0x19, + RS_OP_LITERAL_26 = 0x1a, + RS_OP_LITERAL_27 = 0x1b, + RS_OP_LITERAL_28 = 0x1c, + RS_OP_LITERAL_29 = 0x1d, + RS_OP_LITERAL_30 = 0x1e, + RS_OP_LITERAL_31 = 0x1f, + RS_OP_LITERAL_32 = 0x20, + RS_OP_LITERAL_33 = 0x21, + RS_OP_LITERAL_34 = 0x22, + RS_OP_LITERAL_35 = 0x23, + RS_OP_LITERAL_36 = 0x24, + RS_OP_LITERAL_37 = 0x25, + RS_OP_LITERAL_38 = 0x26, + RS_OP_LITERAL_39 = 0x27, + RS_OP_LITERAL_40 = 0x28, + RS_OP_LITERAL_41 = 0x29, + RS_OP_LITERAL_42 = 0x2a, + RS_OP_LITERAL_43 = 0x2b, + RS_OP_LITERAL_44 = 0x2c, + RS_OP_LITERAL_45 = 0x2d, + RS_OP_LITERAL_46 = 0x2e, + RS_OP_LITERAL_47 = 0x2f, + RS_OP_LITERAL_48 = 0x30, + RS_OP_LITERAL_49 = 0x31, + RS_OP_LITERAL_50 = 0x32, + RS_OP_LITERAL_51 = 0x33, + RS_OP_LITERAL_52 = 0x34, + RS_OP_LITERAL_53 = 0x35, + RS_OP_LITERAL_54 = 0x36, + RS_OP_LITERAL_55 = 0x37, + RS_OP_LITERAL_56 = 0x38, + RS_OP_LITERAL_57 = 0x39, + RS_OP_LITERAL_58 = 0x3a, + RS_OP_LITERAL_59 = 0x3b, + RS_OP_LITERAL_60 = 0x3c, + RS_OP_LITERAL_61 = 0x3d, + RS_OP_LITERAL_62 = 0x3e, + RS_OP_LITERAL_63 = 0x3f, + RS_OP_LITERAL_64 = 0x40, + RS_OP_LITERAL_N1 = 0x41, + RS_OP_LITERAL_N2 = 0x42, + RS_OP_LITERAL_N4 = 0x43, + RS_OP_LITERAL_N8 = 0x44, + RS_OP_COPY_N1_N1 = 0x45, + RS_OP_COPY_N1_N2 = 0x46, + RS_OP_COPY_N1_N4 = 0x47, + RS_OP_COPY_N1_N8 = 0x48, + RS_OP_COPY_N2_N1 = 0x49, + RS_OP_COPY_N2_N2 = 0x4a, + RS_OP_COPY_N2_N4 = 0x4b, + RS_OP_COPY_N2_N8 = 0x4c, + RS_OP_COPY_N4_N1 = 0x4d, + RS_OP_COPY_N4_N2 = 0x4e, + RS_OP_COPY_N4_N4 = 0x4f, + RS_OP_COPY_N4_N8 = 0x50, + RS_OP_COPY_N8_N1 = 0x51, + RS_OP_COPY_N8_N2 = 0x52, + RS_OP_COPY_N8_N4 = 0x53, + RS_OP_COPY_N8_N8 = 0x54, + RS_OP_RESERVED_85 = 0x55, + RS_OP_RESERVED_86 = 0x56, + RS_OP_RESERVED_87 = 0x57, + RS_OP_RESERVED_88 = 0x58, + RS_OP_RESERVED_89 = 0x59, + RS_OP_RESERVED_90 = 0x5a, + RS_OP_RESERVED_91 = 0x5b, + RS_OP_RESERVED_92 = 0x5c, + RS_OP_RESERVED_93 = 0x5d, + RS_OP_RESERVED_94 = 0x5e, + RS_OP_RESERVED_95 = 0x5f, + RS_OP_RESERVED_96 = 0x60, + RS_OP_RESERVED_97 = 0x61, + RS_OP_RESERVED_98 = 0x62, + RS_OP_RESERVED_99 = 0x63, + RS_OP_RESERVED_100 = 0x64, + RS_OP_RESERVED_101 = 0x65, + RS_OP_RESERVED_102 = 0x66, + RS_OP_RESERVED_103 = 0x67, + RS_OP_RESERVED_104 = 0x68, + RS_OP_RESERVED_105 = 0x69, + RS_OP_RESERVED_106 = 0x6a, + RS_OP_RESERVED_107 = 0x6b, + RS_OP_RESERVED_108 = 0x6c, + RS_OP_RESERVED_109 = 0x6d, + RS_OP_RESERVED_110 = 0x6e, + RS_OP_RESERVED_111 = 0x6f, + RS_OP_RESERVED_112 = 0x70, + RS_OP_RESERVED_113 = 0x71, + RS_OP_RESERVED_114 = 0x72, + RS_OP_RESERVED_115 = 0x73, + RS_OP_RESERVED_116 = 0x74, + RS_OP_RESERVED_117 = 0x75, + RS_OP_RESERVED_118 = 0x76, + RS_OP_RESERVED_119 = 0x77, + RS_OP_RESERVED_120 = 0x78, + RS_OP_RESERVED_121 = 0x79, + RS_OP_RESERVED_122 = 0x7a, + RS_OP_RESERVED_123 = 0x7b, + RS_OP_RESERVED_124 = 0x7c, + RS_OP_RESERVED_125 = 0x7d, + RS_OP_RESERVED_126 = 0x7e, + RS_OP_RESERVED_127 = 0x7f, + RS_OP_RESERVED_128 = 0x80, + RS_OP_RESERVED_129 = 0x81, + RS_OP_RESERVED_130 = 0x82, + RS_OP_RESERVED_131 = 0x83, + RS_OP_RESERVED_132 = 0x84, + RS_OP_RESERVED_133 = 0x85, + RS_OP_RESERVED_134 = 0x86, + RS_OP_RESERVED_135 = 0x87, + RS_OP_RESERVED_136 = 0x88, + RS_OP_RESERVED_137 = 0x89, + RS_OP_RESERVED_138 = 0x8a, + RS_OP_RESERVED_139 = 0x8b, + RS_OP_RESERVED_140 = 0x8c, + RS_OP_RESERVED_141 = 0x8d, + RS_OP_RESERVED_142 = 0x8e, + RS_OP_RESERVED_143 = 0x8f, + RS_OP_RESERVED_144 = 0x90, + RS_OP_RESERVED_145 = 0x91, + RS_OP_RESERVED_146 = 0x92, + RS_OP_RESERVED_147 = 0x93, + RS_OP_RESERVED_148 = 0x94, + RS_OP_RESERVED_149 = 0x95, + RS_OP_RESERVED_150 = 0x96, + RS_OP_RESERVED_151 = 0x97, + RS_OP_RESERVED_152 = 0x98, + RS_OP_RESERVED_153 = 0x99, + RS_OP_RESERVED_154 = 0x9a, + RS_OP_RESERVED_155 = 0x9b, + RS_OP_RESERVED_156 = 0x9c, + RS_OP_RESERVED_157 = 0x9d, + RS_OP_RESERVED_158 = 0x9e, + RS_OP_RESERVED_159 = 0x9f, + RS_OP_RESERVED_160 = 0xa0, + RS_OP_RESERVED_161 = 0xa1, + RS_OP_RESERVED_162 = 0xa2, + RS_OP_RESERVED_163 = 0xa3, + RS_OP_RESERVED_164 = 0xa4, + RS_OP_RESERVED_165 = 0xa5, + RS_OP_RESERVED_166 = 0xa6, + RS_OP_RESERVED_167 = 0xa7, + RS_OP_RESERVED_168 = 0xa8, + RS_OP_RESERVED_169 = 0xa9, + RS_OP_RESERVED_170 = 0xaa, + RS_OP_RESERVED_171 = 0xab, + RS_OP_RESERVED_172 = 0xac, + RS_OP_RESERVED_173 = 0xad, + RS_OP_RESERVED_174 = 0xae, + RS_OP_RESERVED_175 = 0xaf, + RS_OP_RESERVED_176 = 0xb0, + RS_OP_RESERVED_177 = 0xb1, + RS_OP_RESERVED_178 = 0xb2, + RS_OP_RESERVED_179 = 0xb3, + RS_OP_RESERVED_180 = 0xb4, + RS_OP_RESERVED_181 = 0xb5, + RS_OP_RESERVED_182 = 0xb6, + RS_OP_RESERVED_183 = 0xb7, + RS_OP_RESERVED_184 = 0xb8, + RS_OP_RESERVED_185 = 0xb9, + RS_OP_RESERVED_186 = 0xba, + RS_OP_RESERVED_187 = 0xbb, + RS_OP_RESERVED_188 = 0xbc, + RS_OP_RESERVED_189 = 0xbd, + RS_OP_RESERVED_190 = 0xbe, + RS_OP_RESERVED_191 = 0xbf, + RS_OP_RESERVED_192 = 0xc0, + RS_OP_RESERVED_193 = 0xc1, + RS_OP_RESERVED_194 = 0xc2, + RS_OP_RESERVED_195 = 0xc3, + RS_OP_RESERVED_196 = 0xc4, + RS_OP_RESERVED_197 = 0xc5, + RS_OP_RESERVED_198 = 0xc6, + RS_OP_RESERVED_199 = 0xc7, + RS_OP_RESERVED_200 = 0xc8, + RS_OP_RESERVED_201 = 0xc9, + RS_OP_RESERVED_202 = 0xca, + RS_OP_RESERVED_203 = 0xcb, + RS_OP_RESERVED_204 = 0xcc, + RS_OP_RESERVED_205 = 0xcd, + RS_OP_RESERVED_206 = 0xce, + RS_OP_RESERVED_207 = 0xcf, + RS_OP_RESERVED_208 = 0xd0, + RS_OP_RESERVED_209 = 0xd1, + RS_OP_RESERVED_210 = 0xd2, + RS_OP_RESERVED_211 = 0xd3, + RS_OP_RESERVED_212 = 0xd4, + RS_OP_RESERVED_213 = 0xd5, + RS_OP_RESERVED_214 = 0xd6, + RS_OP_RESERVED_215 = 0xd7, + RS_OP_RESERVED_216 = 0xd8, + RS_OP_RESERVED_217 = 0xd9, + RS_OP_RESERVED_218 = 0xda, + RS_OP_RESERVED_219 = 0xdb, + RS_OP_RESERVED_220 = 0xdc, + RS_OP_RESERVED_221 = 0xdd, + RS_OP_RESERVED_222 = 0xde, + RS_OP_RESERVED_223 = 0xdf, + RS_OP_RESERVED_224 = 0xe0, + RS_OP_RESERVED_225 = 0xe1, + RS_OP_RESERVED_226 = 0xe2, + RS_OP_RESERVED_227 = 0xe3, + RS_OP_RESERVED_228 = 0xe4, + RS_OP_RESERVED_229 = 0xe5, + RS_OP_RESERVED_230 = 0xe6, + RS_OP_RESERVED_231 = 0xe7, + RS_OP_RESERVED_232 = 0xe8, + RS_OP_RESERVED_233 = 0xe9, + RS_OP_RESERVED_234 = 0xea, + RS_OP_RESERVED_235 = 0xeb, + RS_OP_RESERVED_236 = 0xec, + RS_OP_RESERVED_237 = 0xed, + RS_OP_RESERVED_238 = 0xee, + RS_OP_RESERVED_239 = 0xef, + RS_OP_RESERVED_240 = 0xf0, + RS_OP_RESERVED_241 = 0xf1, + RS_OP_RESERVED_242 = 0xf2, + RS_OP_RESERVED_243 = 0xf3, + RS_OP_RESERVED_244 = 0xf4, + RS_OP_RESERVED_245 = 0xf5, + RS_OP_RESERVED_246 = 0xf6, + RS_OP_RESERVED_247 = 0xf7, + RS_OP_RESERVED_248 = 0xf8, + RS_OP_RESERVED_249 = 0xf9, + RS_OP_RESERVED_250 = 0xfa, + RS_OP_RESERVED_251 = 0xfb, + RS_OP_RESERVED_252 = 0xfc, + RS_OP_RESERVED_253 = 0xfd, + RS_OP_RESERVED_254 = 0xfe, + RS_OP_RESERVED_255 = 0xff +}; +/* END OF AUTOGENERATED CODE */ diff --git a/rsync/qrsync.cpp b/rsync/qrsync.cpp new file mode 100644 index 0000000..fe5f1bc --- a/dev/null +++ b/rsync/qrsync.cpp @@ -0,0 +1,110 @@ +#include "qrsync.h" +#include <stdio.h> +extern "C" { +#include "rsync.h" +} +#include <stdlib.h> + +#include <qfile.h> + +static const char *rdiffNewFile = "/tmp/rdiff/result"; +static size_t block_len = RS_DEFAULT_BLOCK_LEN; +static size_t strong_len = RS_DEFAULT_STRONG_LEN; + + +void QRsync::generateSignature( QString baseFile, QString sigFile ) +{ + + if ( QFile::exists( baseFile ) ) { + FILE *basis_file, *sig_file; + rs_result result; + + basis_file = fopen( baseFile.latin1(), "rb" ); + sig_file = fopen( sigFile.latin1(), "wb" ); + + result = rs_sig_file(basis_file, sig_file, block_len, strong_len, 0); + + fclose( basis_file ); + fclose( sig_file ); + if (result != RS_DONE) + printf("error in rdiffGenSig: %d", result ); + } +} + + +void QRsync::generateDiff( QString baseFile, QString sigFile, QString deltaFile ) +{ + if ( QFile::exists( baseFile ) && QFile::exists( sigFile ) ) { + FILE *sig_file, *new_file, *delta_file; + rs_result result; + rs_signature_t *sumset; + + sig_file = fopen(sigFile.latin1(), "rb"); + new_file = fopen(baseFile.latin1(), "rb"); + delta_file = fopen(deltaFile.latin1(), "wb"); + + result = rs_loadsig_file(sig_file, &sumset, 0); + if (result != RS_DONE) { + qDebug( "rdiffGenDiff: loading of sig file failed, error=%d", result ); + } else { + result = rs_build_hash_table(sumset); + if ( result != RS_DONE) { + qDebug( "rdiffGenDiff: building of hash table failed, error=%d", result ); + } else { + result = rs_delta_file(sumset, new_file, delta_file, 0); + if ( result != RS_DONE) { + qDebug( "rdiffGenDiff: writing of diff file failed, error=%d", result ); + } + } + } + + if ( sumset ) + rs_free_sumset( sumset ); + fclose( new_file ); + fclose( delta_file ); + fclose( sig_file ); + + } +} + +void QRsync::applyDiff( QString baseFile, QString deltaFile ) +{ + if ( QFile::exists( baseFile ) && QFile::exists( deltaFile ) ) { + FILE *basis_file, *delta_file, *new_file; + rs_result result; + + basis_file = fopen(baseFile.latin1(), "rb"); + delta_file = fopen(deltaFile.latin1(), "rb"); +#ifdef Q_WS_WIN + new_file = fopen( (baseFile + ".new").latin1(), "wb" ); +#else + new_file = fopen(rdiffNewFile, "wb"); +#endif + + result = rs_patch_file(basis_file, delta_file, new_file, 0); + + fclose( basis_file ); + fclose( delta_file ); + fclose( new_file ); + + if (result != RS_DONE) { + qDebug( "rdiffApplyDiff failed with result %d", result ); + return; + } + + +#ifdef Q_WS_WIN + QDir dir; + QFile backup = baseFile + "~"; + dir.rename( baseFile, backup ); + dir.rename( (baseFile + ".new"), baseFile ); + dir.remove( backup ); +#else + QString cmd = "mv "; + cmd += rdiffNewFile; + cmd += " " + baseFile; + system( cmd.latin1() ); +#endif + } + +} diff --git a/rsync/qrsync.h b/rsync/qrsync.h new file mode 100644 index 0000000..293c0ff --- a/dev/null +++ b/rsync/qrsync.h @@ -0,0 +1,18 @@ +/***** + */ + +#ifndef QRSYNC_H +#define QRSYNC_H + +#include <qstring.h> + +class QRsync +{ +public: + static void generateSignature( QString baseFile, QString sigFile ); + static void generateDiff( QString baseFile, QString sigFile, QString diffFile ); + static void applyDiff( QString baseFile, QString diffFile ); +}; + + +#endif diff --git a/rsync/rdiff.c b/rsync/rdiff.c new file mode 100644 index 0000000..e08095a --- a/dev/null +++ b/rsync/rdiff.c @@ -0,0 +1,358 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* + | .. after a year and a day, mourning is + | dangerous to the survivor and troublesome + | to the dead. + | -- Harold Bloom + */ + +/* + * rdiff.c -- Command-line network-delta tool. + * + * TODO: Add a -z option to gzip/gunzip patches. This would be + * somewhat useful, but more importantly a good test of the streaming + * API. Also add -I for bzip2. + * + * If built with debug support and we have mcheck, then turn it on. + * (Optionally?) + */ + +#include <config_rsync.h> + +#include <stdlib.h> +#include <stdio.h> +#include <fcntl.h> +#include <popt.h> + +#ifdef HAVE_ZLIB_H +#include <zlib.h> +#endif + +#ifdef HAVE_BZLIB_H +#include <bzlib.h> +#endif + +#include "rsync.h" +#include "fileutil.h" +#include "util.h" +#include "trace.h" +#include "isprefix.h" + + +#define PROGRAM "rdiff" + +static size_t block_len = RS_DEFAULT_BLOCK_LEN; +static size_t strong_len = RS_DEFAULT_STRONG_LEN; + +static int show_stats = 0; + +static int bzip2_level = 0; +static int gzip_level = 0; + + +enum { + OPT_GZIP = 1069, OPT_BZIP2 +}; + +extern int rs_roll_paranoia; + +const struct poptOption opts[] = { + { "verbose", 'v', POPT_ARG_NONE, 0, 'v' }, + { "version", 'V', POPT_ARG_NONE, 0, 'V' }, + { "input-size", 'I', POPT_ARG_INT, &rs_inbuflen }, + { "output-size", 'O', POPT_ARG_INT, &rs_outbuflen }, + { "help", '?', POPT_ARG_NONE, 0, 'h' }, + { 0, 'h', POPT_ARG_NONE, 0, 'h' }, + { "block-size", 'b', POPT_ARG_INT, &block_len }, + { "sum-size", 'S', POPT_ARG_INT, &strong_len }, + { "statistics", 's', POPT_ARG_NONE, &show_stats }, + { "stats", 0, POPT_ARG_NONE, &show_stats }, + { "gzip", 0, POPT_ARG_NONE, 0, OPT_GZIP }, + { "bzip2", 0, POPT_ARG_NONE, 0, OPT_BZIP2 }, + { "paranoia", 0, POPT_ARG_NONE, &rs_roll_paranoia }, + { 0 } +}; + + +static void rdiff_usage(const char *error) +{ + fprintf(stderr, "%s\n" + "Try `%s --help' for more information.\n", + error, PROGRAM); +} + + +static void rdiff_no_more_args(poptContext opcon) +{ + if (poptGetArg(opcon)) { + rdiff_usage("rdiff: too many arguments"); + exit(RS_SYNTAX_ERROR); + } +} + + +static void bad_option(poptContext opcon, int error) +{ + char msgbuf[1000]; + + snprintf(msgbuf, sizeof msgbuf-1, "%s: %s: %s", + PROGRAM, poptStrerror(error), poptBadOption(opcon, 0)); + rdiff_usage(msgbuf); + + exit(RS_SYNTAX_ERROR); +} + + +static void help(void) { + printf("Usage: rdiff [OPTIONS] signature [BASIS [SIGNATURE]]\n" + " [OPTIONS] delta SIGNATURE [NEWFILE [DELTA]]\n" + " [OPTIONS] patch BASIS [DELTA [NEWFILE]]\n" + "\n" + "Options:\n" + " -v, --verbose Trace internal processing\n" + " -V, --version Show program version\n" + " -?, --help Show this help message\n" + " -s, --statistics Show performance statistics\n" + "Delta-encoding options:\n" + " -b, --block-size=BYTES Signature block size\n" + " -S, --sum-size=BYTES Set signature strength\n" + " --paranoia Verify all rolling checksums\n" + "IO options:\n" + " -I, --input-size=BYTES Input buffer size\n" + " -O, --output-size=BYTES Output buffer size\n" + " -z, --gzip[=LEVEL] gzip-compress deltas\n" + " -i, --bzip2[=LEVEL] bzip2-compress deltas\n" + ); +} + + +static void rdiff_show_version(void) +{ + /* + * This little declaration is dedicated to Stephen Kapp and Reaper + * Technologies, who by all appearances redistributed a modified but + * unacknowledged version of GNU Keyring in violation of the licence + * and all laws of politeness and good taste. + */ + char const *bzlib = "", *zlib = "", *trace = ""; + +#ifdef HAVE_LIBZ + zlib = ", gzip"; +#endif + +#ifdef HAVE_LIBBZ2 + bzlib = ", bzip2"; +#endif + +#ifndef DO_RS_TRACE + trace = ", trace disabled"; +#endif + + printf("rdiff (%s) [%s]\n" + "Copyright (C) 1997-2001 by Martin Pool, Andrew Tridgell and others.\n" + "http://rproxy.samba.org/\n" + "Capabilities: %d bit files%s%s%s\n" + "\n" + "librsync comes with NO WARRANTY, to the extent permitted by law.\n" + "You may redistribute copies of librsync under the terms of the GNU\n" + "Lesser General Public License. For more information about these\n" + "matters, see the files named COPYING.\n", + rs_librsync_version, RS_CANONICAL_HOST, + 8 * sizeof(rs_long_t), zlib, bzlib, trace); +} + + + +static void rdiff_options(poptContext opcon) +{ + int c; + char const *a; + + while ((c = poptGetNextOpt(opcon)) != -1) { + switch (c) { + case 'h': + help(); + exit(RS_DONE); + case 'V': + rdiff_show_version(); + exit(RS_DONE); + case 'v': + if (!rs_supports_trace()) { + rs_error("library does not support trace"); + } + rs_trace_set_level(RS_LOG_DEBUG); + break; + + case OPT_GZIP: + case OPT_BZIP2: + if ((a = poptGetOptArg(opcon))) { + int l = atoi(a); + if (c == OPT_GZIP) + gzip_level = l; + else + bzip2_level = l; + } else { + if (c == OPT_GZIP) + gzip_level = -1; /* library default */ + else + bzip2_level = 9; /* demand the best */ + } + rs_error("sorry, compression is not really implemented yet"); + exit(RS_UNIMPLEMENTED); + + default: + bad_option(opcon, c); + } + } +} + + +/** + * Generate signature from remaining command line arguments. + */ +static rs_result rdiff_sig(poptContext opcon) +{ + FILE *basis_file, *sig_file; + rs_stats_t stats; + rs_result result; + + basis_file = rs_file_open(poptGetArg(opcon), "rb"); + sig_file = rs_file_open(poptGetArg(opcon), "wb"); + + rdiff_no_more_args(opcon); + + result = rs_sig_file(basis_file, sig_file, block_len, strong_len, &stats); + if (result != RS_DONE) + return result; + + if (show_stats) + rs_log_stats(&stats); + + return result; +} + + +static rs_result rdiff_delta(poptContext opcon) +{ + FILE *sig_file, *new_file, *delta_file; + char const *sig_name; + rs_result result; + rs_signature_t *sumset; + rs_stats_t stats; + + if (!(sig_name = poptGetArg(opcon))) { + rdiff_usage("Usage for delta: " + "rdiff [OPTIONS] delta SIGNATURE [NEWFILE [DELTA]]"); + return RS_SYNTAX_ERROR; + } + + sig_file = rs_file_open(sig_name, "rb"); + new_file = rs_file_open(poptGetArg(opcon), "rb"); + delta_file = rs_file_open(poptGetArg(opcon), "wb"); + + rdiff_no_more_args(opcon); + + result = rs_loadsig_file(sig_file, &sumset, &stats); + if (result != RS_DONE) + return result; + + if (show_stats) + rs_log_stats(&stats); + + if ((result = rs_build_hash_table(sumset)) != RS_DONE) + return result; + + result = rs_delta_file(sumset, new_file, delta_file, &stats); + + if (show_stats) + rs_log_stats(&stats); + + return result; +} + + + +static rs_result rdiff_patch(poptContext opcon) +{ + /* patch BASIS [DELTA [NEWFILE]] */ + FILE *basis_file, *delta_file, *new_file; + char const *basis_name; + rs_stats_t stats; + rs_result result; + + if (!(basis_name = poptGetArg(opcon))) { + rdiff_usage("Usage for patch: " + "rdiff [OPTIONS] patch BASIS [DELTA [NEW]]"); + return RS_SYNTAX_ERROR; + } + + basis_file = rs_file_open(basis_name, "rb"); + delta_file = rs_file_open(poptGetArg(opcon), "rb"); + new_file = rs_file_open(poptGetArg(opcon), "wb"); + + rdiff_no_more_args(opcon); + + result = rs_patch_file(basis_file, delta_file, new_file, &stats); + + if (show_stats) + rs_log_stats(&stats); + + return result; +} + + + +static rs_result rdiff_action(poptContext opcon) +{ + const char *action; + + action = poptGetArg(opcon); + if (!action) + ; + else if (isprefix(action, "signature")) + return rdiff_sig(opcon); + else if (isprefix(action, "delta")) + return rdiff_delta(opcon); + else if (isprefix(action, "patch")) + return rdiff_patch(opcon); + + rdiff_usage("rdiff: You must specify an action: `signature', `delta', or `patch'."); + return RS_SYNTAX_ERROR; +} + + +int main(const int argc, const char *argv[]) +{ + poptContext opcon; + rs_result result; + + opcon = poptGetContext(PROGRAM, argc, argv, opts, 0); + rdiff_options(opcon); + result = rdiff_action(opcon); + + if (result != RS_DONE) + rs_log(RS_LOG_ERR|RS_LOG_NONAME, "%s", rs_strerror(result)); + + return result; +} diff --git a/rsync/readsums.c b/rsync/readsums.c new file mode 100644 index 0000000..21b5ecd --- a/dev/null +++ b/rsync/readsums.c @@ -0,0 +1,214 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell <tridge@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +/* + * readsums.c -- Load signatures from a file into an ::rs_signature_t. + */ + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> + +#include "rsync.h" +#include "sumset.h" +#include "job.h" +#include "trace.h" +#include "netint.h" +#include "protocol.h" +#include "util.h" +#include "stream.h" + + +static rs_result rs_loadsig_s_weak(rs_job_t *job); +static rs_result rs_loadsig_s_strong(rs_job_t *job); + + + +/** + * Add a just-read-in checksum pair to the signature block. + */ +static rs_result rs_loadsig_add_sum(rs_job_t *job, rs_strong_sum_t *strong) +{ + size_t new_size; + rs_signature_t *sig = job->signature; + rs_block_sig_t *asignature; + + sig->count++; + new_size = sig->count * sizeof(rs_block_sig_t); + + sig->block_sigs = realloc(sig->block_sigs, new_size); + + if (sig->block_sigs == NULL) { + return RS_MEM_ERROR; + } + asignature = &(sig->block_sigs[sig->count - 1]); + + asignature->weak_sum = job->weak_sig; + asignature->i = sig->count; + + memcpy(asignature->strong_sum, strong, sig->strong_sum_len); + + if (rs_trace_enabled()) { + char hexbuf[RS_MD4_LENGTH * 2 + 2]; + rs_hexify(hexbuf, strong, sig->strong_sum_len); + + rs_trace("read in checksum: weak=%#x, strong=%s", asignature->weak_sum, + hexbuf); + } + + job->stats.sig_blocks++; + + return RS_RUNNING; +} + + +static rs_result rs_loadsig_s_weak(rs_job_t *job) +{ + int l; + rs_result result; + + result = rs_suck_n4(job, &l); + if (result == RS_DONE) + ; + else if (result == RS_INPUT_ENDED) /* ending here is OK */ + return RS_DONE; + else + return result; + + job->weak_sig = l; + + job->statefn = rs_loadsig_s_strong; + + return RS_RUNNING; +} + + + +static rs_result rs_loadsig_s_strong(rs_job_t *job) +{ + rs_result result; + rs_strong_sum_t *strongsum; + + result = rs_scoop_read(job, job->signature->strong_sum_len, + (void **) &strongsum); + if (result != RS_DONE) return result; + + job->statefn = rs_loadsig_s_weak; + + return rs_loadsig_add_sum(job, strongsum); +} + + + +static rs_result rs_loadsig_s_stronglen(rs_job_t *job) +{ + int l; + rs_result result; + + if ((result = rs_suck_n4(job, &l)) != RS_DONE) + return result; + job->strong_sum_len = l; + + if (l < 0 || l > RS_MD4_LENGTH) { + rs_error("strong sum length %d is implausible", l); + return RS_CORRUPT; + } + + job->signature->block_len = job->block_len; + job->signature->strong_sum_len = job->strong_sum_len; + + rs_trace("allocated sigset_t (strong_sum_len=%d, block_len=%d)", + (int) job->strong_sum_len, (int) job->block_len); + + job->statefn = rs_loadsig_s_weak; + + return RS_RUNNING; +} + + +static rs_result rs_loadsig_s_blocklen(rs_job_t *job) +{ + int l; + rs_result result; + + if ((result = rs_suck_n4(job, &l)) != RS_DONE) + return result; + job->block_len = l; + + if (job->block_len < 1) { + rs_error("block length of %d is bogus", (int) job->block_len); + return RS_CORRUPT; + } + + job->statefn = rs_loadsig_s_stronglen; + job->stats.block_len = job->block_len; + + return RS_RUNNING; +} + + +static rs_result rs_loadsig_s_magic(rs_job_t *job) +{ + int l; + rs_result result; + + if ((result = rs_suck_n4(job, &l)) != RS_DONE) { + return result; + } else if (l != RS_SIG_MAGIC) { + rs_error("wrong magic number %#10x for signature", l); + return RS_BAD_MAGIC; + } else { + rs_trace("got signature magic %#10x", l); + } + + job->statefn = rs_loadsig_s_blocklen; + + return RS_RUNNING; +} + + +/** + * \brief Read a signature from a file into an ::rs_signature_t structure + * in memory. + * + * Once there, it can be used to generate a delta to a newer version of + * the file. + * + * \note After loading the signatures, you must call + * rs_build_hash_table() before you can use them. + */ +rs_job_t *rs_loadsig_begin(rs_signature_t **signature) +{ + rs_job_t *job; + + job = rs_job_new("loadsig", rs_loadsig_s_magic); + *signature = job->signature = rs_alloc_struct(rs_signature_t); + job->signature->count = 0; + + return job; +} + diff --git a/rsync/rsync.h b/rsync/rsync.h new file mode 100644 index 0000000..a80a368 --- a/dev/null +++ b/rsync/rsync.h @@ -0,0 +1,388 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* + | You should never wear your best + | trousers when you go out to fight for + | freedom and liberty. + | -- Henrik Ibsen + */ + + +/** \file rsync.h + * + * \brief Main public interface to librsync. + * \author Martin Pool <mbp@samba.org> + * \version librsync-0.9.1 + * + * $Id$ + * + * See \ref intro for an introduction to use of this library. + */ + +#include <sys/types.h> + +extern char const rs_librsync_version[]; +extern char const rs_licence_string[]; + + +/** + * \brief A long integer type that can handle the largest file + * offsets. + * + * Perhaps this might have to be configured to be 'long long', 'long', + * or something else depending on the platform. + */ +#if SIZEOF_LONG_LONG +typedef signed long long rs_long_t; +#else +typedef long rs_long_t; +#endif + + +/** + * \brief Log severity levels. + * + * These are the same as syslog, at least in glibc. + * + * \sa rs_trace_set_level() + */ +typedef enum { + RS_LOG_EMERG = 0, /**< System is unusable */ + RS_LOG_ALERT = 1, /**< Action must be taken immediately */ + RS_LOG_CRIT = 2, /**< Critical conditions */ + RS_LOG_ERR = 3, /**< Error conditions */ + RS_LOG_WARNING = 4, /**< Warning conditions */ + RS_LOG_NOTICE = 5, /**< Normal but significant condition */ + RS_LOG_INFO = 6, /**< Informational */ + RS_LOG_DEBUG = 7 /**< Debug-level messages */ +} rs_loglevel; + + + +/** + * \typedef rs_trace_fn_t + * \brief Callback to write out log messages. + * \param level a syslog level. + * \param msg message to be logged. + */ +typedef void rs_trace_fn_t(int level, char const *msg); + +void rs_trace_set_level(rs_loglevel level); + +/** Set trace callback. */ +void rs_trace_to(rs_trace_fn_t *); + +/** Default trace callback that writes to stderr. Implements + * ::rs_trace_fn_t, and may be passed to rs_trace_to(). */ +void rs_trace_stderr(int level, char const *msg); + +/** Check whether the library was compiled with debugging trace + * suport. */ +int rs_supports_trace(void); + + + +/** + * Convert FROM_LEN bytes at FROM_BUF into a hex representation in + * TO_BUF, which must be twice as long plus one byte for the null + * terminator. + */ +void rs_hexify(char *to_buf, void const *from_buf, int from_len); + +/** + * Decode a base64 buffer in place. \return the number of binary + * bytes. + */ +size_t rs_unbase64(char *s); + + +/** + * Encode a buffer as base64. + */ +void rs_base64(unsigned char const *buf, int n, char *out); + + +/** + * \brief Return codes from nonblocking rsync operations. + */ +typedef enum { + RS_DONE = 0, /**< Completed successfully. */ + RS_BLOCKED = 1, /**< Blocked waiting for more data. */ + RS_RUNNING = 2, /**< Not yet finished or blocked. + * This value should never be returned + * to the caller. */ + + RS_TEST_SKIPPED = 77, /**< Test neither passed or failed. */ + + RS_IO_ERROR = 100, /**< Error in file or network IO. */ + RS_SYNTAX_ERROR = 101, /**< Command line syntax error. */ + RS_MEM_ERROR = 102, /**< Out of memory. */ + RS_INPUT_ENDED = 103, /**< End of input file, possibly + unexpected. */ + RS_BAD_MAGIC = 104, /**< Bad magic number at start of + stream. Probably not a librsync + file, or possibly the wrong kind of + file or from an incompatible + library version. */ + RS_UNIMPLEMENTED = 105, /**< Author is lazy. */ + RS_CORRUPT = 106, /**< Unbelievable value in stream. */ + RS_INTERNAL_ERROR = 107, /**< Probably a library bug. */ + RS_PARAM_ERROR = 108 /**< Bad value passed in to library, + * probably an application bug. */ +} rs_result; + + + +/** + * Return an English description of a ::rs_result value. + */ +char const *rs_strerror(rs_result r); + + +/** + * \brief Performance statistics from a librsync encoding or decoding + * operation. + * + * \sa rs_format_stats(), rs_log_stats() + */ +typedef struct rs_stats { + char const *op; /**< Human-readable name of current + * operation. For example, "delta". */ + int lit_cmds; /**< Number of literal commands. */ + rs_long_t lit_bytes; /**< Number of literal bytes. */ + rs_long_t lit_cmdbytes; /**< Number of bytes used in literal + * command headers. */ + + rs_long_t copy_cmds, copy_bytes, copy_cmdbytes; + rs_long_t sig_cmds, sig_bytes; + int false_matches; + + rs_long_t sig_blocks; /**< Number of blocks described by the + signature. */ + + size_t block_len; + + rs_long_t in_bytes; /**< Total bytes read from input. */ + rs_long_t out_bytes; /**< Total bytes written to output. */ +} rs_stats_t; + + +/** \typedef struct rs_mdfour rs_mdfour_t + * + * \brief MD4 message-digest accumulator. + * + * \sa rs_mdfour(), rs_mdfour_begin(), rs_mdfour_update(), + * rs_mdfour_result() + */ +typedef struct rs_mdfour { + int A, B, C, D; + int totalN; + int tail_len; + unsigned char tail[64]; +} rs_mdfour_t; + +#define RS_MD4_LENGTH 16 + +typedef unsigned int rs_weak_sum_t; +typedef unsigned char rs_strong_sum_t[RS_MD4_LENGTH]; + +void rs_mdfour(unsigned char *out, void const *in, size_t); +void rs_mdfour_begin(/* @out@ */ rs_mdfour_t * md); +void rs_mdfour_update(rs_mdfour_t * md, void const *, + size_t n); +void rs_mdfour_result(rs_mdfour_t * md, unsigned char *out); + +char *rs_format_stats(rs_stats_t const *, char *, size_t); + +int rs_log_stats(rs_stats_t const *stats); + + +typedef struct rs_signature rs_signature_t; + +void rs_free_sumset(rs_signature_t *); +void rs_sumset_dump(rs_signature_t const *); + + +/** + * Stream through which the calling application feeds data to and from the + * library. + * + * On each call to rs_job_iter, the caller can make available + * + * - avail_in bytes of input data at next_in + * - avail_out bytes of output space at next_out + * - some of both + * + * Buffers must be allocated and passed in by the caller. This + * routine never allocates, reallocates or frees buffers. + * + * Pay attention to the meaning of the returned pointer and length + * values. They do \b not indicate the location and amount of + * returned data. Rather, if \p *out_ptr was originally set to \p + * out_buf, then the output data begins at \p out_buf, and has length + * \p *out_ptr - \p out_buf. + * + * Note also that if \p *avail_in is nonzero on return, then not all of + * the input data has been consumed. The caller should either provide + * more output buffer space and call rs_work() again passing the same + * \p next_in and \p avail_in, or put the remaining input data into some + * persistent buffer and call rs_work() with it again when there is + * more output space. + * + * \param next_in References a pointer which on entry should point to + * the start of the data to be encoded. Updated to point to the byte + * after the last one consumed. + * + * \param avail_in References the length of available input. Updated to + * be the number of unused data bytes, which will be zero if all the + * input was consumed. May be zero if there is no new input, but the + * caller just wants to drain output. + * + * \param next_out References a pointer which on entry points to the + * start of the output buffer. Updated to point to the byte after the + * last one filled. + * + * \param avail_out References the size of available output buffer. + * Updated to the size of unused output buffer. + * + * \return The ::rs_result that caused iteration to stop. + * + * \sa rs_buffers_t + * \sa \ref api_buffers + */ +struct rs_buffers_s { + char *next_in; /**< Next input byte */ + size_t avail_in; /**< Number of bytes available at next_in */ + int eof_in; /**< True if there is no more data + * after this. */ + + char *next_out; /**< Next output byte should be put there */ + size_t avail_out; /**< Remaining free space at next_out */ +}; + +/** + * Stream through which the calling application feeds data to and from the + * library. + * + * \sa struct rs_buffers_s + * \sa \ref api_buffers + */ +typedef struct rs_buffers_s rs_buffers_t; + +/** Default length of strong signatures, in bytes. The MD4 checksum + * is truncated to this size. */ +#define RS_DEFAULT_STRONG_LEN 8 + +/** Default block length, if not determined by any other factors. */ +#define RS_DEFAULT_BLOCK_LEN 2048 + + +/** \typedef struct rs_job rs_job_t + * + * \brief Job of work to be done. + * + * Created by functions such as rs_sig_begin(), and then iterated + * over by rs_job_iter(). */ +typedef struct rs_job rs_job_t; + +/** + * Bitmask values that may be passed to the options parameter of + * rs_work(). + */ +typedef enum rs_work_options { + RS_END = 0x01 /**< End of input file; please finish + * up. */ +} rs_work_options; + + +rs_result rs_job_iter(rs_job_t *, rs_buffers_t *); + +typedef rs_result rs_driven_cb(rs_job_t *job, rs_buffers_t *buf, + void *opaque); + +rs_result rs_job_drive(rs_job_t *job, rs_buffers_t *buf, + rs_driven_cb in_cb, void *in_opaque, + rs_driven_cb out_cb, void *out_opaque); + +rs_result rs_job_free(rs_job_t *); + +int rs_accum_value(rs_job_t *, char *sum, size_t sum_len); + +rs_job_t *rs_sig_begin(size_t new_block_len, size_t strong_sum_len); + +rs_job_t *rs_delta_begin(rs_signature_t *); + +rs_job_t *rs_loadsig_begin(rs_signature_t **); + +/** + * \brief Callback used to retrieve parts of the basis file. + * + * \param pos Position where copying should begin. + * + * \param len On input, the amount of data that should be retrieved. + * Updated to show how much is actually available. + * + * \param buf On input, a buffer of at least \p *len bytes. May be + * updated to point to a buffer allocated by the callback if it + * prefers. + */ +typedef rs_result rs_copy_cb(void *opaque, off_t pos, + size_t *len, void **buf); + + + +rs_job_t *rs_patch_begin(rs_copy_cb *, void *copy_arg); + + +rs_result rs_build_hash_table(rs_signature_t* sums); + + + +#ifndef RSYNC_NO_STDIO_INTERFACE +/** + * Buffer sizes for file IO. + * + * You probably only need to change these in testing. + */ +extern int rs_inbuflen, rs_outbuflen; + + +/** + * Calculate the MD4 sum of a file. + * + * \param result Binary (not hex) MD4 of the whole contents of the + * file. + */ +void rs_mdfour_file(FILE *in_file, char *result); + +rs_result rs_sig_file(FILE *old_file, FILE *sig_file, + size_t block_len, size_t strong_len, rs_stats_t *); + +rs_result rs_loadsig_file(FILE *, rs_signature_t **, rs_stats_t *); + +rs_result rs_file_copy_cb(void *arg, off_t pos, size_t *len, void **buf); + +rs_result rs_delta_file(rs_signature_t *, FILE *new_file, FILE *delta_file, rs_stats_t *); + +rs_result rs_patch_file(FILE *basis_file, FILE *delta_file, FILE *new_file, rs_stats_t *); +#endif diff --git a/rsync/scoop.c b/rsync/scoop.c new file mode 100644 index 0000000..9f68a60 --- a/dev/null +++ b/rsync/scoop.c @@ -0,0 +1,271 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +/* + * scoop.c -- This file deals with readahead from caller-supplied + * buffers. + * + * Many functions require a certain minimum amount of input to do their + * processing. For example, to calculate a strong checksum of a block + * we need at least a block of input. + * + * Since we put the buffers completely under the control of the caller, + * we can't count on ever getting this much data all in one go. We + * can't simply wait, because the caller might have a smaller buffer + * than we require and so we'll never get it. For the same reason we + * must always accept all the data we're given. + * + * So, stream input data that's required for readahead is put into a + * special buffer, from which the caller can then read. It's + * essentially like an internal pipe, which on any given read request + * may or may not be able to actually supply the data. + * + * As a future optimization, we might try to take data directly from the + * input buffer if there's already enough there. + */ + +/* + * TODO: We probably know a maximum amount of data that can be scooped + * up, so we could just avoid dynamic allocation. However that can't + * be fixed at compile time, because when generating a delta it needs + * to be large enough to hold one full block. Perhaps we can set it + * up when the job is allocated? It would be kind of nice to not do + * any memory allocation after startup, as bzlib does this. + */ + + + /* + | To walk on water you've gotta sink + | in the ice. + | -- Shihad, `The General Electric'. + */ + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> + +#include "rsync.h" +#include "job.h" +#include "stream.h" +#include "trace.h" +#include "util.h" + + +#if 0 +# undef rs_trace +# define rs_trace(s...) +#endif + + +/** + * Try to accept a from the input buffer to get LEN bytes in the scoop. + */ +void rs_scoop_input(rs_job_t *job, size_t len) +{ + rs_buffers_t *stream = job->stream; + size_t tocopy; + + assert(len > job->scoop_avail); + + if (job->scoop_alloc < len) { + /* need to allocate a new buffer, too */ + char *newbuf; + int newsize = 2 * len; + newbuf = rs_alloc(newsize, "scoop buffer"); + if (job->scoop_avail) + memcpy(newbuf, job->scoop_next, job->scoop_avail); + if (job->scoop_buf) + free(job->scoop_buf); + job->scoop_buf = job->scoop_next = newbuf; + rs_trace("resized scoop buffer to %.0f bytes from %.0f", + (double) newsize, (double) job->scoop_alloc); + job->scoop_alloc = newsize; + } else { + /* this buffer size is fine, but move the existing + * data down to the front. */ + memmove(job->scoop_buf, job->scoop_next, job->scoop_avail); + job->scoop_next = job->scoop_buf; + } + + /* take as much input as is available, to give up to LEN bytes + * in the scoop. */ + tocopy = len - job->scoop_avail; + if (tocopy > stream->avail_in) + tocopy = stream->avail_in; + assert(tocopy + job->scoop_avail <= job->scoop_alloc); + + memcpy(job->scoop_next + job->scoop_avail, stream->next_in, tocopy); + rs_trace("accepted %.0f bytes from input to scoop", (double) tocopy); + job->scoop_avail += tocopy; + stream->next_in += tocopy; + stream->avail_in -= tocopy; +} + + +/** + * Advance the input cursor forward \p len bytes. This is used after + * doing readahead, when you decide you want to keep it. \p len must + * be no more than the amount of available data, so you can't cheat. + * + * So when creating a delta, we require one block of readahead. But + * after examining that block, we might decide to advance over all of + * it (if there is a match), or just one byte (if not). + */ +void rs_scoop_advance(rs_job_t *job, size_t len) +{ + rs_buffers_t *stream = job->stream; + + /* It never makes sense to advance over a mixture of bytes from + * the scoop and input, because you couldn't possibly have looked + * at them all at the same time. */ + if (job->scoop_avail) { + /* reading from the scoop buffer */ + rs_trace("advance over %d bytes from scoop", len); + assert(len <= job->scoop_avail); + job->scoop_avail -= len; + job->scoop_next += len; + } else { + rs_trace("advance over %d bytes from input buffer", len); + assert(len <= stream->avail_in); + stream->avail_in -= len; + stream->next_in += len; + } +} + + + +/** + * \brief Read from scoop without advancing. + * + * Ask for LEN bytes of input from the stream. If that much data is + * available, then return a pointer to it in PTR, advance the stream + * input pointer over the data, and return RS_DONE. If there's not + * enough data, then accept whatever is there into a buffer, advance + * over it, and return RS_BLOCKED. + * + * The data is not actually removed from the input, so this function + * lets you do readahead. If you want to keep any of the data, you + * should also call rs_scoop_advance() to skip over it. + */ +rs_result rs_scoop_readahead(rs_job_t *job, size_t len, void **ptr) +{ + rs_buffers_t *stream = job->stream; + rs_job_check(job); + + if (job->scoop_avail >= len) { + /* We have enough data queued to satisfy the request, + * so go straight from the scoop buffer. */ + rs_trace("got %.0f bytes direct from scoop", (double) len); + *ptr = job->scoop_next; + return RS_DONE; + } else if (job->scoop_avail) { + /* We have some data in the scoop, but not enough to + * satisfy the request. */ + rs_trace("data is present in the scoop and must be used"); + rs_scoop_input(job, len); + + if (job->scoop_avail < len) { + rs_trace("still have only %.0f bytes in scoop", + (double) job->scoop_avail); + return RS_BLOCKED; + } else { + rs_trace("scoop now has %.0f bytes, this is enough", + (double) job->scoop_avail); + *ptr = job->scoop_next; + return RS_DONE; + } + } else if (stream->avail_in >= len) { + /* There's enough data in the stream's input */ + *ptr = stream->next_in; + rs_trace("got %.0f bytes from input buffer", (double) len); + return RS_DONE; + } else if (stream->avail_in > 0) { + /* Nothing was queued before, but we don't have enough + * data to satisfy the request. So queue what little + * we have, and try again next time. */ + rs_trace("couldn't satisfy request for %.0f, scooping %.0f bytes", + (double) len, (double) job->scoop_avail); + rs_scoop_input(job, len); + return RS_BLOCKED; + } else if (stream->eof_in) { + /* Nothing is queued before, and nothing is in the input + * buffer at the moment. */ + rs_trace("reached end of input stream"); + return RS_INPUT_ENDED; + } else { + /* Nothing queued at the moment. */ + rs_trace("blocked with no data in scoop or input buffer"); + return RS_BLOCKED; + } +} + + + +/** + * Read LEN bytes if possible, and remove them from the input scoop. + * If there's not enough data yet, return RS_BLOCKED. + * + * \param ptr will be updated to point to a read-only buffer holding + * the data, if enough is available. + * + * \return RS_DONE if all the data was available, RS_BLOCKED if it's + * not there. + */ +rs_result rs_scoop_read(rs_job_t *job, size_t len, void **ptr) +{ + rs_result result; + + result = rs_scoop_readahead(job, len, ptr); + if (result == RS_DONE) + rs_scoop_advance(job, len); + + return result; +} + + + +/* + * Read whatever remains in the input stream, assuming that it runs up + * to the end of the file. Set LEN appropriately. + */ +rs_result rs_scoop_read_rest(rs_job_t *job, size_t *len, void **ptr) +{ + rs_buffers_t *stream = job->stream; + + *len = job->scoop_avail + stream->avail_in; + + return rs_scoop_read(job, *len, ptr); +} + + + +/** + * Return the total number of bytes available including the scoop and input + * buffer. + */ +size_t rs_scoop_total_avail(rs_job_t *job) +{ + return job->scoop_avail + job->stream->avail_in; +} diff --git a/rsync/search.c b/rsync/search.c new file mode 100644 index 0000000..3e0c5e2 --- a/dev/null +++ b/rsync/search.c @@ -0,0 +1,162 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +/* + * This file contains code for searching the sumset for matching + * values. + */ + +/* + * TODO: The common case is that the next block in both streams + * match. Can we make that a bit faster at all? We'd need to perhaps + * add a link forward between blocks in the sum_struct corresponding + * to the order they're found in the input; then before doing a search + * we can just check that pointer. + */ + +#include <config_rsync.h> + +#include <string.h> +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" +#include "trace.h" +#include "util.h" +#include "sumset.h" +#include "search.h" +#include "checksum.h" + + +#define TABLESIZE (1<<16) +#define NULL_TAG (-1) + + +#define gettag2(s1,s2) (((s1) + (s2)) & 0xFFFF) +#define gettag(sum) gettag2((sum)&0xFFFF,(sum)>>16) + + +static int +rs_compare_targets(rs_target_t const *t1, rs_target_t const *t2) +{ + return ((int) t1->t - (int) t2->t); +} + + +rs_result +rs_build_hash_table(rs_signature_t * sums) +{ + int i; + + sums->tag_table = calloc(TABLESIZE, sizeof sums->tag_table[0]); + if (!sums->tag_table) + return RS_MEM_ERROR; + + if (sums->count > 0) { + sums->targets = calloc(sums->count, sizeof(rs_target_t)); + if (!sums->targets) + return RS_MEM_ERROR; + + for (i = 0; i < sums->count; i++) { + sums->targets[i].i = i; + sums->targets[i].t = gettag(sums->block_sigs[i].weak_sum); + } + + /* FIXME: Perhaps if this operating system has comparison_fn_t + * like GNU, then use it in the cast. But really does anyone + * care? */ + qsort(sums->targets, sums->count, + sizeof(sums->targets[0]), + (int (*)(const void *, const void *)) rs_compare_targets); + } + + for (i = 0; i < TABLESIZE; i++) + sums->tag_table[i] = NULL_TAG; + + for (i = sums->count - 1; i >= 0; i--) { + sums->tag_table[sums->targets[i].t] = i; + } + + rs_trace("done"); + return RS_DONE; +} + + + +/* + * See if there is a match for the specified block INBUF..BLOCK_LEN in + * the checksum set, using precalculated WEAK_SUM. + * + * If we don't find a match on the weak checksum, then we just give + * up. If we do find a weak match, then we proceed to calculate the + * strong checksum for the current block, and see if it will match + * anything. + */ +int +rs_search_for_block(rs_weak_sum_t weak_sum, + char const *inbuf, size_t block_len, + rs_signature_t const *sig, rs_stats_t * stats, + rs_long_t * match_where) +{ + int hash_tag = gettag(weak_sum); + int j = sig->tag_table[hash_tag]; + rs_strong_sum_t strong_sum; + int got_strong = 0; + + if (j == NULL_TAG) { + return 0; + } + + for (; j < sig->count && sig->targets[j].t == hash_tag; j++) { + int i = sig->targets[j].i; + int token; + + if (weak_sum != sig->block_sigs[i].weak_sum) + continue; + + token = sig->block_sigs[i].i; + + rs_trace("found weak match for %08x in token %d", weak_sum, token); + + if (!got_strong) { + rs_calc_strong_sum(inbuf, block_len, &strong_sum); + got_strong = 1; + } + + /* FIXME: Use correct dynamic sum length! */ + if (memcmp(strong_sum, sig->block_sigs[i].strong_sum, + sig->strong_sum_len) == 0) { + /* XXX: This is a remnant of rsync: token number 1 is the + * block at offset 0. It would be good to clear this + * up. */ + *match_where = (token - 1) * sig->block_len; + return 1; + } else { + rs_trace("this was a false positive, the strong sig doesn't match"); + stats->false_matches++; + } + } + + return 0; +} diff --git a/rsync/search.h b/rsync/search.h new file mode 100644 index 0000000..a1d0cd8 --- a/dev/null +++ b/rsync/search.h @@ -0,0 +1,29 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +int +rs_search_for_block(rs_weak_sum_t weak_sum, + char const *inbuf, size_t block_len, + rs_signature_t const *sums, rs_stats_t * stats, + rs_long_t * match_where); + diff --git a/rsync/snprintf.c b/rsync/snprintf.c new file mode 100644 index 0000000..348ef2d --- a/dev/null +++ b/rsync/snprintf.c @@ -0,0 +1,822 @@ +/* $Id$ */ + +/* + * Copyright Patrick Powell 1995 + * This code is based on code written by Patrick Powell (papowell@astart.com) + * It may be used for any purpose as long as this notice remains intact + * on all source code distributions + */ + +/************************************************************** + * Original: + * Patrick Powell Tue Apr 11 09:48:21 PDT 1995 + * A bombproof version of doprnt (dopr) included. + * Sigh. This sort of thing is always nasty do deal with. Note that + * the version here does not include floating point... + * + * snprintf() is used instead of sprintf() as it does limit checks + * for string length. This covers a nasty loophole. + * + * The other functions are there to prevent NULL pointers from + * causing nast effects. + * + * More Recently: + * Brandon Long <blong@fiction.net> 9/15/96 for mutt 0.43 + * This was ugly. It is still ugly. I opted out of floating point + * numbers, but the formatter understands just about everything + * from the normal C string format, at least as far as I can tell from + * the Solaris 2.5 printf(3S) man page. + * + * Brandon Long <blong@fiction.net> 10/22/97 for mutt 0.87.1 + * Ok, added some minimal floating point support, which means this + * probably requires libm on most operating systems. Don't yet + * support the exponent (e,E) and sigfig (g,G). Also, fmtint() + * was pretty badly broken, it just wasn't being exercised in ways + * which showed it, so that's been fixed. Also, formated the code + * to mutt conventions, and removed dead code left over from the + * original. Also, there is now a builtin-test, just compile with: + * gcc -DTEST_SNPRINTF -o snprintf snprintf.c -lm + * and run snprintf for results. + * + * Thomas Roessler <roessler@guug.de> 01/27/98 for mutt 0.89i + * The PGP code was using unsigned hexadecimal formats. + * Unfortunately, unsigned formats simply didn't work. + * + * Michael Elkins <me@cs.hmc.edu> 03/05/98 for mutt 0.90.8 + * The original code assumed that both snprintf() and vsnprintf() were + * missing. Some systems only have snprintf() but not vsnprintf(), so + * the code is now broken down under HAVE_SNPRINTF and HAVE_VSNPRINTF. + * + * Andrew Tridgell <tridge@samba.org> Oct 1998 + * fixed handling of %.0f + * added test for HAVE_LONG_DOUBLE + * + **************************************************************/ + +#include <config_rsync.h> + +#include <string.h> +# include <ctype.h> +#include <sys/types.h> + +#if !defined(HAVE_SNPRINTF) || !defined(HAVE_VSNPRINTF) + +/* Define this as a fall through, HAVE_STDARG_H is probably already set */ + +#define HAVE_VARARGS_H + +/* varargs declarations: */ + +#if defined(HAVE_STDARG_H) +# include <stdarg.h> +# define HAVE_STDARGS /* let's hope that works everywhere (mj) */ +# define VA_LOCAL_DECL va_list ap +# define VA_START(f) va_start(ap, f) +# define VA_SHIFT(v,t) ; /* no-op for ANSI */ +# define VA_END va_end(ap) +#else +# if defined(HAVE_VARARGS_H) +# include <varargs.h> +# undef HAVE_STDARGS +# define VA_LOCAL_DECL va_list ap +# define VA_START(f) va_start(ap) /* f is ignored! */ +# define VA_SHIFT(v,t) v = va_arg(ap,t) +# define VA_END va_end(ap) +# else +/*XX ** NO VARARGS ** XX*/ +# endif +#endif + +#ifdef HAVE_LONG_DOUBLE +#define LDOUBLE long double +#else +#define LDOUBLE double +#endif + +/*int snprintf (char *str, size_t count, const char *fmt, ...);*/ +/*int vsnprintf (char *str, size_t count, const char *fmt, va_list arg);*/ + +static void dopr (char *buffer, size_t maxlen, const char *format, + va_list args); +static void fmtstr (char *buffer, size_t *currlen, size_t maxlen, + char *value, int flags, int min, int max); +static void fmtint (char *buffer, size_t *currlen, size_t maxlen, + long value, int base, int min, int max, int flags); +static void fmtfp (char *buffer, size_t *currlen, size_t maxlen, + LDOUBLE fvalue, int min, int max, int flags); +static void dopr_outch (char *buffer, size_t *currlen, size_t maxlen, char c ); + +/* + * dopr(): poor man's version of doprintf + */ + +/* format read states */ +#define DP_S_DEFAULT 0 +#define DP_S_FLAGS 1 +#define DP_S_MIN 2 +#define DP_S_DOT 3 +#define DP_S_MAX 4 +#define DP_S_MOD 5 +#define DP_S_CONV 6 +#define DP_S_DONE 7 + +/* format flags - Bits */ +#define DP_F_MINUS (1 << 0) +#define DP_F_PLUS (1 << 1) +#define DP_F_SPACE (1 << 2) +#define DP_F_NUM (1 << 3) +#define DP_F_ZERO (1 << 4) +#define DP_F_UP (1 << 5) +#define DP_F_UNSIGNED (1 << 6) + +/* Conversion Flags */ +#define DP_C_SHORT 1 +#define DP_C_LONG 2 +#define DP_C_LDOUBLE 3 + +#define char_to_int(p) (p - '0') +#define MAX(p,q) ((p >= q) ? p : q) + +static void dopr (char *buffer, size_t maxlen, const char *format, va_list args) +{ + char ch; + long value; + LDOUBLE fvalue; + char *strvalue; + int min; + int max; + int state; + int flags; + int cflags; + size_t currlen; + + state = DP_S_DEFAULT; + currlen = flags = cflags = min = 0; + max = -1; + ch = *format++; + + while (state != DP_S_DONE) + { + if ((ch == '\0') || (currlen >= maxlen)) + state = DP_S_DONE; + + switch(state) + { + case DP_S_DEFAULT: + if (ch == '%') + state = DP_S_FLAGS; + else + dopr_outch (buffer, &currlen, maxlen, ch); + ch = *format++; + break; + case DP_S_FLAGS: + switch (ch) + { + case '-': + flags |= DP_F_MINUS; + ch = *format++; + break; + case '+': + flags |= DP_F_PLUS; + ch = *format++; + break; + case ' ': + flags |= DP_F_SPACE; + ch = *format++; + break; + case '#': + flags |= DP_F_NUM; + ch = *format++; + break; + case '0': + flags |= DP_F_ZERO; + ch = *format++; + break; + default: + state = DP_S_MIN; + break; + } + break; + case DP_S_MIN: + if (isdigit((unsigned char)ch)) + { + min = 10*min + char_to_int (ch); + ch = *format++; + } + else if (ch == '*') + { + min = va_arg (args, int); + ch = *format++; + state = DP_S_DOT; + } + else + state = DP_S_DOT; + break; + case DP_S_DOT: + if (ch == '.') + { + state = DP_S_MAX; + ch = *format++; + } + else + state = DP_S_MOD; + break; + case DP_S_MAX: + if (isdigit((unsigned char)ch)) + { + if (max < 0) + max = 0; + max = 10*max + char_to_int (ch); + ch = *format++; + } + else if (ch == '*') + { + max = va_arg (args, int); + ch = *format++; + state = DP_S_MOD; + } + else + state = DP_S_MOD; + break; + case DP_S_MOD: + /* Currently, we don't support Long Long, bummer */ + switch (ch) + { + case 'h': + cflags = DP_C_SHORT; + ch = *format++; + break; + case 'l': + cflags = DP_C_LONG; + ch = *format++; + break; + case 'L': + cflags = DP_C_LDOUBLE; + ch = *format++; + break; + default: + break; + } + state = DP_S_CONV; + break; + case DP_S_CONV: + switch (ch) + { + case 'd': + case 'i': + if (cflags == DP_C_SHORT) + value = va_arg (args, short int); + else if (cflags == DP_C_LONG) + value = va_arg (args, long int); + else + value = va_arg (args, int); + fmtint (buffer, &currlen, maxlen, value, 10, min, max, flags); + break; + case 'o': + flags |= DP_F_UNSIGNED; + if (cflags == DP_C_SHORT) + value = va_arg (args, unsigned short int); + else if (cflags == DP_C_LONG) + value = (long)va_arg (args, unsigned long int); + else + value = (long)va_arg (args, unsigned int); + fmtint (buffer, &currlen, maxlen, value, 8, min, max, flags); + break; + case 'u': + flags |= DP_F_UNSIGNED; + if (cflags == DP_C_SHORT) + value = va_arg (args, unsigned short int); + else if (cflags == DP_C_LONG) + value = (long)va_arg (args, unsigned long int); + else + value = (long)va_arg (args, unsigned int); + fmtint (buffer, &currlen, maxlen, value, 10, min, max, flags); + break; + case 'X': + flags |= DP_F_UP; + case 'x': + flags |= DP_F_UNSIGNED; + if (cflags == DP_C_SHORT) + value = va_arg (args, unsigned short int); + else if (cflags == DP_C_LONG) + value = (long)va_arg (args, unsigned long int); + else + value = (long)va_arg (args, unsigned int); + fmtint (buffer, &currlen, maxlen, value, 16, min, max, flags); + break; + case 'f': + if (cflags == DP_C_LDOUBLE) + fvalue = va_arg (args, LDOUBLE); + else + fvalue = va_arg (args, double); + /* um, floating point? */ + fmtfp (buffer, &currlen, maxlen, fvalue, min, max, flags); + break; + case 'E': + flags |= DP_F_UP; + case 'e': + if (cflags == DP_C_LDOUBLE) + fvalue = va_arg (args, LDOUBLE); + else + fvalue = va_arg (args, double); + break; + case 'G': + flags |= DP_F_UP; + case 'g': + if (cflags == DP_C_LDOUBLE) + fvalue = va_arg (args, LDOUBLE); + else + fvalue = va_arg (args, double); + break; + case 'c': + dopr_outch (buffer, &currlen, maxlen, va_arg (args, int)); + break; + case 's': + strvalue = va_arg (args, char *); + if (max < 0) + max = maxlen; /* ie, no max */ + fmtstr (buffer, &currlen, maxlen, strvalue, flags, min, max); + break; + case 'p': + strvalue = va_arg (args, void *); + fmtint (buffer, &currlen, maxlen, (long) strvalue, 16, min, max, flags); + break; + case 'n': + if (cflags == DP_C_SHORT) + { + short int *num; + num = va_arg (args, short int *); + *num = currlen; + } + else if (cflags == DP_C_LONG) + { + long int *num; + num = va_arg (args, long int *); + *num = (long int)currlen; + } + else + { + int *num; + num = va_arg (args, int *); + *num = currlen; + } + break; + case '%': + dopr_outch (buffer, &currlen, maxlen, ch); + break; + case 'w': + /* not supported yet, treat as next char */ + ch = *format++; + break; + default: + /* Unknown, skip */ + break; + } + ch = *format++; + state = DP_S_DEFAULT; + flags = cflags = min = 0; + max = -1; + break; + case DP_S_DONE: + break; + default: + /* hmm? */ + break; /* some picky compilers need this */ + } + } + if (currlen < maxlen - 1) + buffer[currlen] = '\0'; + else + buffer[maxlen - 1] = '\0'; +} + +static void fmtstr (char *buffer, size_t *currlen, size_t maxlen, + char *value, int flags, int min, int max) +{ + int padlen, strln; /* amount to pad */ + int cnt = 0; + + if (value == 0) + { + value = "<NULL>"; + } + + for (strln = 0; value[strln]; ++strln); /* strlen */ + padlen = min - strln; + if (padlen < 0) + padlen = 0; + if (flags & DP_F_MINUS) + padlen = -padlen; /* Left Justify */ + + while ((padlen > 0) && (cnt < max)) + { + dopr_outch (buffer, currlen, maxlen, ' '); + --padlen; + ++cnt; + } + while (*value && (cnt < max)) + { + dopr_outch (buffer, currlen, maxlen, *value++); + ++cnt; + } + while ((padlen < 0) && (cnt < max)) + { + dopr_outch (buffer, currlen, maxlen, ' '); + ++padlen; + ++cnt; + } +} + +/* Have to handle DP_F_NUM (ie 0x and 0 alternates) */ + +static void fmtint (char *buffer, size_t *currlen, size_t maxlen, + long value, int base, int min, int max, int flags) +{ + int signvalue = 0; + unsigned long uvalue; + char convert[20]; + int place = 0; + int spadlen = 0; /* amount to space pad */ + int zpadlen = 0; /* amount to zero pad */ + int caps = 0; + + if (max < 0) + max = 0; + + uvalue = value; + + if(!(flags & DP_F_UNSIGNED)) + { + if( value < 0 ) { + signvalue = '-'; + uvalue = -value; + } + else + if (flags & DP_F_PLUS) /* Do a sign (+/i) */ + signvalue = '+'; + else + if (flags & DP_F_SPACE) + signvalue = ' '; + } + + if (flags & DP_F_UP) caps = 1; /* Should characters be upper case? */ + + do { + convert[place++] = + (caps? "0123456789ABCDEF":"0123456789abcdef") + [uvalue % (unsigned)base ]; + uvalue = (uvalue / (unsigned)base ); + } while(uvalue && (place < 20)); + if (place == 20) place--; + convert[place] = 0; + + zpadlen = max - place; + spadlen = min - MAX (max, place) - (signvalue ? 1 : 0); + if (zpadlen < 0) zpadlen = 0; + if (spadlen < 0) spadlen = 0; + if (flags & DP_F_ZERO) + { + zpadlen = MAX(zpadlen, spadlen); + spadlen = 0; + } + if (flags & DP_F_MINUS) + spadlen = -spadlen; /* Left Justifty */ + +#ifdef DEBUG_SNPRINTF + printf("zpad: %d, spad: %d, min: %d, max: %d, place: %d\n", + zpadlen, spadlen, min, max, place); +#endif + + /* Spaces */ + while (spadlen > 0) + { + dopr_outch (buffer, currlen, maxlen, ' '); + --spadlen; + } + + /* Sign */ + if (signvalue) + dopr_outch (buffer, currlen, maxlen, signvalue); + + /* Zeros */ + if (zpadlen > 0) + { + while (zpadlen > 0) + { + dopr_outch (buffer, currlen, maxlen, '0'); + --zpadlen; + } + } + + /* Digits */ + while (place > 0) + dopr_outch (buffer, currlen, maxlen, convert[--place]); + + /* Left Justified spaces */ + while (spadlen < 0) { + dopr_outch (buffer, currlen, maxlen, ' '); + ++spadlen; + } +} + +static LDOUBLE abs_val (LDOUBLE value) +{ + LDOUBLE result = value; + + if (value < 0) + result = -value; + + return result; +} + +static LDOUBLE pow10 (int exp) +{ + LDOUBLE result = 1; + + while (exp) + { + result *= 10; + exp--; + } + + return result; +} + +static long round (LDOUBLE value) +{ + long intpart; + + intpart = (long)value; + value = value - intpart; + if (value >= 0.5) + intpart++; + + return intpart; +} + +static void fmtfp (char *buffer, size_t *currlen, size_t maxlen, + LDOUBLE fvalue, int min, int max, int flags) +{ + int signvalue = 0; + LDOUBLE ufvalue; + char iconvert[20]; + char fconvert[20]; + int iplace = 0; + int fplace = 0; + int padlen = 0; /* amount to pad */ + int zpadlen = 0; + int caps = 0; + long intpart; + long fracpart; + + /* + * AIX manpage says the default is 0, but Solaris says the default + * is 6, and sprintf on AIX defaults to 6 + */ + if (max < 0) + max = 6; + + ufvalue = abs_val (fvalue); + + if (fvalue < 0) + signvalue = '-'; + else + if (flags & DP_F_PLUS) /* Do a sign (+/i) */ + signvalue = '+'; + else + if (flags & DP_F_SPACE) + signvalue = ' '; + +#if 0 + if (flags & DP_F_UP) caps = 1; /* Should characters be upper case? */ +#endif + + intpart = (long)ufvalue; + + /* + * Sorry, we only support 9 digits past the decimal because of our + * conversion method + */ + if (max > 9) + max = 9; + + /* We "cheat" by converting the fractional part to integer by + * multiplying by a factor of 10 + */ + fracpart = round ((pow10 (max)) * (ufvalue - intpart)); + + if (fracpart >= pow10 (max)) + { + intpart++; + fracpart -= pow10 (max); + } + +#ifdef DEBUG_SNPRINTF + printf("fmtfp: %g %d.%d min=%d max=%d\n", + (double)fvalue, intpart, fracpart, min, max); +#endif + + /* Convert integer part */ + do { + iconvert[iplace++] = + (caps? "0123456789ABCDEF":"0123456789abcdef")[intpart % 10]; + intpart = (intpart / 10); + } while(intpart && (iplace < 20)); + if (iplace == 20) iplace--; + iconvert[iplace] = 0; + + /* Convert fractional part */ + do { + fconvert[fplace++] = + (caps? "0123456789ABCDEF":"0123456789abcdef")[fracpart % 10]; + fracpart = (fracpart / 10); + } while(fracpart && (fplace < 20)); + if (fplace == 20) fplace--; + fconvert[fplace] = 0; + + /* -1 for decimal point, another -1 if we are printing a sign */ + padlen = min - iplace - max - 1 - ((signvalue) ? 1 : 0); + zpadlen = max - fplace; + if (zpadlen < 0) + zpadlen = 0; + if (padlen < 0) + padlen = 0; + if (flags & DP_F_MINUS) + padlen = -padlen; /* Left Justifty */ + + if ((flags & DP_F_ZERO) && (padlen > 0)) + { + if (signvalue) + { + dopr_outch (buffer, currlen, maxlen, signvalue); + --padlen; + signvalue = 0; + } + while (padlen > 0) + { + dopr_outch (buffer, currlen, maxlen, '0'); + --padlen; + } + } + while (padlen > 0) + { + dopr_outch (buffer, currlen, maxlen, ' '); + --padlen; + } + if (signvalue) + dopr_outch (buffer, currlen, maxlen, signvalue); + + while (iplace > 0) + dopr_outch (buffer, currlen, maxlen, iconvert[--iplace]); + + +#ifdef DEBUG_SNPRINTF + printf("fmtfp: fplace=%d zpadlen=%d\n", fplace, zpadlen); +#endif + + /* + * Decimal point. This should probably use locale to find the correct + * char to print out. + */ + if (max > 0) { + dopr_outch (buffer, currlen, maxlen, '.'); + + while (fplace > 0) + dopr_outch (buffer, currlen, maxlen, fconvert[--fplace]); + } + + while (zpadlen > 0) + { + dopr_outch (buffer, currlen, maxlen, '0'); + --zpadlen; + } + + while (padlen < 0) + { + dopr_outch (buffer, currlen, maxlen, ' '); + ++padlen; + } +} + +static void dopr_outch (char *buffer, size_t *currlen, size_t maxlen, char c) +{ + if (*currlen < maxlen) + buffer[(*currlen)++] = c; +} +#endif /* !defined(HAVE_SNPRINTF) || !defined(HAVE_VSNPRINTF) */ + +#ifndef HAVE_VSNPRINTF + int vsnprintf (char *str, size_t count, const char *fmt, va_list args) +{ + str[0] = 0; + dopr(str, count, fmt, args); + return(strlen(str)); +} +#endif /* !HAVE_VSNPRINTF */ + +#ifndef HAVE_SNPRINTF +/* VARARGS3 */ +#ifdef HAVE_STDARGS + int snprintf (char *str,size_t count,const char *fmt,...) +#else + int snprintf (va_alist) va_dcl +#endif +{ +#ifndef HAVE_STDARGS + char *str; + size_t count; + char *fmt; +#endif + VA_LOCAL_DECL; + + VA_START (fmt); + VA_SHIFT (str, char *); + VA_SHIFT (count, size_t ); + VA_SHIFT (fmt, char *); + (void) vsnprintf(str, count, fmt, ap); + VA_END; + return(strlen(str)); +} + + +#else + /* keep compilers happy about empty files */ + void dummy_snprintf(void) {} +#endif /* !HAVE_SNPRINTF */ + +#ifdef TEST_SNPRINTF +#ifndef LONG_STRING +#define LONG_STRING 1024 +#endif + int main (void) +{ + char buf1[LONG_STRING]; + char buf2[LONG_STRING]; + char *fp_fmt[] = { + "%-1.5f", + "%1.5f", + "%123.9f", + "%10.5f", + "% 10.5f", + "%+22.9f", + "%+4.9f", + "%01.3f", + "%4f", + "%3.1f", + "%3.2f", + "%.0f", + "%.1f", + NULL + }; + double fp_nums[] = { -1.5, 134.21, 91340.2, 341.1234, 0203.9, 0.96, 0.996, + 0.9996, 1.996, 4.136, 0}; + char *int_fmt[] = { + "%-1.5d", + "%1.5d", + "%123.9d", + "%5.5d", + "%10.5d", + "% 10.5d", + "%+22.33d", + "%01.3d", + "%4d", + NULL + }; + long int_nums[] = { -1, 134, 91340, 341, 0203, 0}; + int x, y; + int fail = 0; + int num = 0; + + printf ("Testing snprintf format codes against system sprintf...\n"); + + for (x = 0; fp_fmt[x] != NULL ; x++) + for (y = 0; fp_nums[y] != 0 ; y++) + { + snprintf (buf1, sizeof (buf1), fp_fmt[x], fp_nums[y]); + sprintf (buf2, fp_fmt[x], fp_nums[y]); + if (strcmp (buf1, buf2)) + { + printf("snprintf doesn't match Format: %s\n\tsnprintf = %s\n\tsprintf = %s\n", + fp_fmt[x], buf1, buf2); + fail++; + } + num++; + } + + for (x = 0; int_fmt[x] != NULL ; x++) + for (y = 0; int_nums[y] != 0 ; y++) + { + snprintf (buf1, sizeof (buf1), int_fmt[x], int_nums[y]); + sprintf (buf2, int_fmt[x], int_nums[y]); + if (strcmp (buf1, buf2)) + { + printf("snprintf doesn't match Format: %s\n\tsnprintf = %s\n\tsprintf = %s\n", + int_fmt[x], buf1, buf2); + fail++; + } + num++; + } + printf ("%d tests failed out of %d.\n", fail, num); +} +#endif /* SNPRINTF_TEST */ + diff --git a/rsync/stats.c b/rsync/stats.c new file mode 100644 index 0000000..5767d52 --- a/dev/null +++ b/rsync/stats.c @@ -0,0 +1,114 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <sys/file.h> +#include <string.h> + +#include "rsync.h" +#include "trace.h" + +/* + * TODO: Other things to show in statistics: + * + * Number of input and output bytes. + * + * Number of times we blocked waiting for input or output. + * + * Number of blocks. + */ + +int +rs_log_stats(rs_stats_t const *stats) +{ + char buf[1000]; + + rs_format_stats(stats, buf, sizeof buf - 1); + rs_log(RS_LOG_INFO|RS_LOG_NONAME, "%s", buf); + return 0; +} + + + +/** + * \brief Return a human-readable representation of statistics. + * + * The string is truncated if it does not fit. 100 characters should + * be sufficient space. + * + * \param stats Statistics from an encoding or decoding operation. + * + * \param buf Buffer to receive result. + * \param size Size of buffer. + * \return buf + */ +char * +rs_format_stats(rs_stats_t const * stats, + char *buf, size_t size) +{ + char const *op = stats->op; + int len; + + if (!op) + op = "noop"; + + len = snprintf(buf, size, "%s statistics: ", op); + + if (stats->lit_cmds) { + len += snprintf(buf+len, size-len, + "literal[%d cmds, %.0f bytes, %.0f cmdbytes] ", + stats->lit_cmds, + (double) stats->lit_bytes, + (double) stats->lit_cmdbytes); + } + + if (stats->sig_cmds) { + len += snprintf(buf+len, size-len, + "in-place-signature[%.0f cmds, %.0f bytes] ", + (double) stats->sig_cmds, + (double) stats->sig_bytes); + } + + if (stats->copy_cmds || stats->false_matches) { + len += snprintf(buf+len, size-len, + "copy[%.0f cmds, %.0f bytes, %.0f false, %.0f cmdbytes]", + (double) stats->copy_cmds, + (double) stats->copy_bytes, + (double) stats->false_matches, + (double) stats->copy_cmdbytes); + } + + + if (stats->sig_blocks) { + len += snprintf(buf+len, size-len, + "signature[%.0f blocks, %.0f bytes per block]", + (double) stats->sig_blocks, + (double) stats->block_len); + } + + return buf; +} diff --git a/rsync/stream.c b/rsync/stream.c new file mode 100644 index 0000000..d20d866 --- a/dev/null +++ b/rsync/stream.c @@ -0,0 +1,155 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- dynamic caching and delta update in HTTP + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* + * Programming languages should be designed not + * by piling feature on top of feature, but by + * removing the weaknesses and restrictions that + * make additional features appear necessary. + * -- Revised^5 Report on Scheme + */ + + +/* + * OK, so I'll admit IO here is a little complex. The most important + * player here is the stream, which is an object for managing filter + * operations. It has both input and output sides, both of which is + * just a (pointer,len) pair into a buffer provided by the client. + * The code controlling the stream handles however much data it wants, + * and the client provides or accepts however much is convenient. + * + * At the same time as being friendly to the client, we also try to be + * very friendly to the internal code. It wants to be able to ask for + * arbitrary amounts of input or output and get it without having to + * keep track of partial completion. So there are functions which + * either complete, or queue whatever was not sent and return + * RS_BLOCKED. + * + * The output buffer is a little more clever than simply a data + * buffer. Instead it knows that we can send either literal data, or + * data copied through from the input of the stream. + * + * In buf.c you will find functions that then map buffers onto stdio + * files. + * + * So on return from an encoding function, either the input or the + * output or possibly both will have no more bytes available. + */ + +/* + * Manage librsync streams of IO. See scoop.c and tube.c for related + * code for input and output respectively. + * + * librsync never does IO or memory allocation, but relies on the + * caller. This is very nice for integration, but means that we have + * to be fairly flexible as to when we can `read' or `write' stuff + * internally. + * + * librsync basically does two types of IO. It reads network integers + * of various lengths which encode command and control information + * such as versions and signatures. It also does bulk data transfer. + * + * IO of network integers is internally buffered, because higher + * levels of the code need to see them transmitted atomically: it's no + * good to read half of a uint32. So there is a small and fixed + * length internal buffer which accumulates these. Unlike previous + * versions of the library, we don't require that the caller hold the + * start until the whole thing has arrived, which guarantees that we + * can always make progress. + * + * On each call into a stream iterator, it should begin by trying to + * flush output. This may well use up all the remaining stream space, + * in which case nothing else can be done. + */ + +/* TODO: Return errors rather than aborting if something goes wrong. */ + + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +#include "rsync.h" +#include "stream.h" +#include "util.h" +#include "trace.h" + +static const int RS_STREAM_DOGTAG = 2001125; + + +/** + * \brief Copy up to \p max_len bytes from input of \b stream to its output. + * + * Return the number of bytes actually copied, which may be less than + * LEN if there is not enough space in one or the other stream. + * + * This always does the copy immediately. Most functions should call + * rs_tube_copy() to cause the copy to happen gradually as space + * becomes available. + */ +int rs_buffers_copy(rs_buffers_t *stream, int max_len) +{ + int len = max_len; + + assert(len > 0); + + if ((unsigned) len > stream->avail_in) { + rs_trace("copy limited to %d available input bytes", + stream->avail_in); + len = stream->avail_in; + } + + + if ((unsigned) len > stream->avail_out) { + rs_trace("copy limited to %d available output bytes", + stream->avail_out); + len = stream->avail_out; + } + + if (!len) + return 0; +/* rs_trace("stream copied chunk of %d bytes", len); */ + + memcpy(stream->next_out, stream->next_in, len); + + stream->next_out += len; + stream->avail_out -= len; + + stream->next_in += len; + stream->avail_in -= len; + + return len; +} + + +/** + * Whenever a stream processing function exits, it should have done so + * because it has either consumed all the input or has filled the + * output buffer. This function checks that simple postcondition. + */ +void rs_buffers_check_exit(rs_buffers_t const *stream) +{ + assert(stream->avail_in == 0 || stream->avail_out == 0); +} diff --git a/rsync/stream.h b/rsync/stream.h new file mode 100644 index 0000000..e2d0da8 --- a/dev/null +++ b/rsync/stream.h @@ -0,0 +1,46 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* Two wars in a lifetime bear hard on the little places. + * In winter when storms come rushing out of the dark, + * And the bay boils like a cauldron of sharks, + * The old remember the trenches at Paschendale + * And sons who died on the Burma Railway. */ + + +int rs_buffers_is_empty(rs_buffers_t *stream); +int rs_buffers_copy(rs_buffers_t *stream, int len); + +int rs_tube_catchup(rs_job_t *); +void rs_tube_write(rs_job_t *, void const *buf, size_t len); +void rs_tube_copy(rs_job_t *, int len); +int rs_tube_is_idle(rs_job_t const *); +void rs_check_tube(rs_job_t *); + +void rs_buffers_check_exit(rs_buffers_t const *); + +void rs_scoop_advance(rs_job_t *, size_t len); +rs_result rs_scoop_readahead(rs_job_t *, size_t len, void **ptr); +rs_result rs_scoop_read(rs_job_t *, size_t len, void **ptr); +rs_result rs_scoop_read_rest(rs_job_t *, size_t *len, void **ptr); +size_t rs_scoop_total_avail(rs_job_t *job); +void rs_scoop_input(rs_job_t *job, size_t len); diff --git a/rsync/sumset.c b/rsync/sumset.c new file mode 100644 index 0000000..c03aeb4 --- a/dev/null +++ b/rsync/sumset.c @@ -0,0 +1,83 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <stdio.h> + +#include "rsync.h" +#include "sumset.h" +#include "util.h" +#include "trace.h" + + +/** + * Deep deallocation of checksums. + */ +void +rs_free_sumset(rs_signature_t * psums) +{ + if (psums->block_sigs) + free(psums->block_sigs); + + if (psums->tag_table) + free(psums->tag_table); + + if (psums->targets) + free(psums->targets); + + rs_bzero(psums, sizeof *psums); + free(psums); +} + + + +/** + * Dump signatures to the log. + */ +void +rs_sumset_dump(rs_signature_t const *sums) +{ + int i; + char strong_hex[RS_MD4_LENGTH * 3]; + + rs_log(RS_LOG_INFO, + "sumset info: block_len=%d, file length=%lu, " + "number of chunks=%d, remainder=%d", + sums->block_len, + (unsigned long) sums->flength, sums->count, + sums->remainder); + + for (i = 0; i < sums->count; i++) { + rs_hexify(strong_hex, sums->block_sigs[i].strong_sum, + sums->strong_sum_len); + rs_log(RS_LOG_INFO, + "sum %6d: weak=%08x, strong=%s", + i, sums->block_sigs[i].weak_sum, strong_hex); + } +} + + + diff --git a/rsync/sumset.h b/rsync/sumset.h new file mode 100644 index 0000000..a501b23 --- a/dev/null +++ b/rsync/sumset.h @@ -0,0 +1,67 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell <tridge@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +/* + * TODO: These structures are not terribly useful. Perhaps we need a + * splay tree or something that will let us smoothly grow as data is + * read in. + */ + + +/** + * \brief Description of the match described by a signature. + */ +typedef struct rs_target { + unsigned short t; + int i; +} rs_target_t; + +typedef struct rs_block_sig rs_block_sig_t; + +/* + * This structure describes all the sums generated for an instance of + * a file. It incorporates some redundancy to make it easier to + * search. + */ +struct rs_signature { + rs_long_t flength; /* total file length */ + int count; /* how many chunks */ + int remainder; /* flength % block_length */ + int block_len; /* block_length */ + int strong_sum_len; + rs_block_sig_t *block_sigs; /* points to info for each chunk */ + int *tag_table; + rs_target_t *targets; +}; + + +/* + * All blocks are the same length in the current algorithm except for + * the last block which may be short. + */ +struct rs_block_sig { + int i; /* index of this chunk */ + rs_weak_sum_t weak_sum; /* simple checksum */ + rs_strong_sum_t strong_sum; /* checksum */ +}; diff --git a/rsync/trace.c b/rsync/trace.c new file mode 100644 index 0000000..b7e2b87 --- a/dev/null +++ b/rsync/trace.c @@ -0,0 +1,225 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* + | Finality is death. + | Perfection is finality. + | Nothing is perfect. + | There are lumps in it. + */ + + + +/* + * TODO: Have a bit set in the log level that says not to include the + * function name. + */ + +#include <config_rsync.h> + +#include <unistd.h> +#include <stdio.h> +#include <sys/file.h> +#include <string.h> +#include <errno.h> +#include <stdlib.h> +#include <assert.h> +#include <stdarg.h> + +#include "rsync.h" +#include "util.h" +#include "trace.h" + + +rs_trace_fn_t *rs_trace_impl = rs_trace_stderr; + +int rs_trace_level = RS_LOG_INFO; + +#ifdef HAVE_PROGRAM_INVOCATION_NAME +# define MY_NAME program_invocation_short_name +#else +# define MY_NAME "librsync" +#endif + +static void rs_log_va(int level, char const *fn, char const *fmt, va_list va); + +#if SIZEOF_SIZE_T > SIZEOF_LONG +# warning size_t is larger than a long integer, values in trace messages may be wrong +#endif + + +/** + * Log severity strings, if any. Must match ordering in + * ::rs_loglevel. + */ +static const char *rs_severities[] = { + "EMERGENCY! ", "ALERT! ", "CRITICAL! ", "ERROR: ", "Warning: ", + "", "", "" +}; + + + +/** + * \brief Set the destination of trace information. + * + * The callback scheme allows for use within applications that may + * have their own particular ways of reporting errors: log files for a + * web server, perhaps, and an error dialog for a browser. + * + * \todo Do we really need such fine-grained control, or just yes/no + * tracing? + */ +void +rs_trace_to(rs_trace_fn_t * new_impl) +{ + rs_trace_impl = new_impl; +} + + +/** + * Set the least important message severity that will be output. + */ +void +rs_trace_set_level(rs_loglevel level) +{ + rs_trace_level = level; +} + + +static void +rs_log_va(int flags, char const *fn, char const *fmt, va_list va) +{ + int level = flags & RS_LOG_PRIMASK; + + if (rs_trace_impl && level <= rs_trace_level) { + char buf[1000]; + char full_buf[1000]; + + vsnprintf(buf, sizeof buf - 1, fmt, va); + + if (flags & RS_LOG_NONAME) { + snprintf(full_buf, sizeof full_buf - 1, + "%s: %s%s\n", + MY_NAME, rs_severities[level], buf); + } else { + snprintf(full_buf, sizeof full_buf - 1, + "%s: %s(%s) %s\n", + MY_NAME, rs_severities[level], fn, buf); + } + + rs_trace_impl(level, full_buf); + } +} + + + +/** + * Called by a macro, used on platforms where we can't determine the + * calling function name. + */ +void +rs_log0_nofn(int level, char const *fmt, ...) +{ + va_list va; + + va_start(va, fmt); + rs_log_va(level, PACKAGE, fmt, va); + va_end(va); +} + + +/* Called by a macro that prepends the calling function name, + * etc. */ +void +rs_log0(int level, char const *fn, char const *fmt, ...) +{ + va_list va; + + va_start(va, fmt); + rs_log_va(level, fn, fmt, va); + va_end(va); +} + + +void +rs_trace_stderr(int UNUSED(level), char const *msg) +{ + /* NOTE NO TRAILING NUL */ + write(STDERR_FILENO, msg, strlen(msg)); +} + + +/* This is called directly if the machine doesn't allow varargs + * macros. */ +void +rs_fatal0(char const *s, ...) +{ + va_list va; + + va_start(va, s); + rs_log_va(RS_LOG_CRIT, PACKAGE, s, va); + va_end(va); +} + + +/* This is called directly if the machine doesn't allow varargs + * macros. */ +void +rs_error0(char const *s, ...) +{ + va_list va; + + va_start(va, s); + rs_log_va(RS_LOG_ERR, PACKAGE, s, va); + va_end(va); +} + + +/* This is called directly if the machine doesn't allow varargs + * macros. */ +void +rs_trace0(char const *s, ...) +{ + va_list va; + + va_start(va, s); + rs_log_va(RS_LOG_DEBUG, PACKAGE, s, va); + va_end(va); +} + + +/** + * Return true if the library contains trace code; otherwise false. + * If this returns false, then trying to turn trace on will achieve + * nothing. + */ +int +rs_supports_trace(void) +{ +#ifdef DO_RS_TRACE + return 1; +#else + return 0; +#endif /* !DO_RS_TRACE */ +} + + diff --git a/rsync/trace.h b/rsync/trace.h new file mode 100644 index 0000000..60a6477 --- a/dev/null +++ b/rsync/trace.h @@ -0,0 +1,122 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- generate and apply network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +/* + * TODO: A function like perror that includes strerror output. Apache + * does this by adding flags as well as the severity level which say + * whether such information should be included. + */ + + +/* + * trace may be turned off. + * + * error is always on, but you can return and continue in some way + * + * fatal terminates the whole process + */ + +void rs_fatal0(char const *s, ...); +void rs_error0(char const *s, ...); +void rs_trace0(char const *s, ...); + +void rs_log0_nofn(int level, char const *fmt, ...); + +#ifdef __GNUC__ + +void rs_log0(int level, char const *fn, char const *fmt, ...) + __attribute__ ((format(printf, 3, 4))); + +#ifdef DO_RS_TRACE +# define rs_trace(fmt, arg...) \ + do { rs_log0(RS_LOG_DEBUG, __FUNCTION__, fmt , ##arg); \ + } while (0) +#else +# define rs_trace(s, str...) +#endif /* !DO_RS_TRACE */ + +/* + * TODO: Don't assume this is a gcc thing; rather test in autoconf for + * support for __FUNCTION__ and varargs macros. One simple way might + * just be to try compiling the definition of one of these functions! + * + * TODO: Also look for the C9X predefined identifier `_function', or + * whatever it's called. + */ + +#define rs_log(l, s, str...) do { \ + rs_log0((l), __FUNCTION__, (s) , ##str); \ + } while (0) + + +#define rs_error(s, str...) do { \ + rs_log0(RS_LOG_ERR, __FUNCTION__, (s) , ##str); \ + } while (0) + + +#define rs_fatal(s, str...) do { \ + rs_log0(RS_LOG_CRIT, __FUNCTION__, \ + (s) , ##str); \ + abort(); \ + } while (0) + + +#else /************************* ! __GNUC__ */ + +# define rs_fatal rs_fatal0 +# define rs_error rs_error0 +# define rs_log rs_log0_nofn + +# ifdef DO_RS_TRACE +# define rs_trace rs_trace0 +# endif /* DO_RS_TRACE */ +#endif /* ! __GNUC__ */ + + +void rs_log0(int level, char const *fn, char const *fmt, ...); + + +enum { + RS_LOG_PRIMASK = 7, /**< Mask to extract priority + part. \internal */ + + RS_LOG_NONAME = 8 /**< \b Don't show function name in + message. */ +}; + + + +/** + * \macro rs_trace_enabled() + * + * Call this before putting too much effort into generating trace + * messages. + */ + +extern int rs_trace_level; + +#ifdef DO_RS_TRACE +# define rs_trace_enabled() ((rs_trace_level & RS_LOG_PRIMASK) >= RS_LOG_DEBUG) +#else +# define rs_trace_enabled() 0 +#endif diff --git a/rsync/tube.c b/rsync/tube.c new file mode 100644 index 0000000..0b82adc --- a/dev/null +++ b/rsync/tube.c @@ -0,0 +1,264 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- dynamic caching and delta update in HTTP + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* + | Where a calculator on the ENIAC is + | equpped with 18,000 vaccuum tubes and + | weighs 30 tons, computers in the + | future may have only 1,000 vaccuum + | tubes and perhaps weigh 1 1/2 + | tons. + | -- Popular Mechanics, March 1949 + */ + + +/* tube: a somewhat elastic but fairly small buffer for data passing + * through a stream. + * + * In most cases the iter can adjust to send just as much data will + * fit. In some cases that would be too complicated, because it has + * to transmit an integer or something similar. So in that case we + * stick whatever won't fit into a small buffer. + * + * A tube can contain some literal data to go out (typically command + * bytes), and also an instruction to copy data from the stream's + * input or from some other location. Both literal data and a copy + * command can be queued at the same time, but only in that order and + * at most one of each. */ + + +/* + * TODO: As an optimization, write it directly to the stream if + * possible. But for simplicity don't do that yet. + * + * TODO: I think our current copy code will lock up if the application + * only ever calls us with either input or output buffers, and not + * both. So I guess in that case we might need to copy into some + * temporary buffer space, and then back out again later. + */ + + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +#include "rsync.h" +#include "trace.h" +#include "util.h" +#include "job.h" +#include "stream.h" + + +static void rs_tube_catchup_write(rs_job_t *job) +{ + rs_buffers_t *stream = job->stream; + int len, remain; + + len = job->write_len; + assert(len > 0); + + assert(len > 0); + if ((size_t) len > stream->avail_out) + len = stream->avail_out; + + if (!stream->avail_out) { + rs_trace("no output space available"); + return; + } + + memcpy(stream->next_out, job->write_buf, len); + stream->next_out += len; + stream->avail_out -= len; + + remain = job->write_len - len; + rs_trace("transmitted %d write bytes from tube, " + "%d remain to be sent", + len, remain); + + if (remain > 0) { + /* Still something left in the tube... */ + memmove(job->write_buf, job->write_buf + len, remain); + } else { + assert(remain == 0); + } + + job->write_len = remain; +} + + +/** + * Execute a copy command, taking data from the scoop. + * + * \sa rs_tube_catchup_copy() + */ +static void +rs_tube_copy_from_scoop(rs_job_t *job) +{ + size_t this_len; + rs_buffers_t *stream = job->stream; + + this_len = job->copy_len; + if (this_len > job->scoop_avail) { + this_len = job->scoop_avail; + } + if (this_len > stream->avail_out) { + this_len = stream->avail_out; + } + + memcpy(stream->next_out, job->scoop_next, this_len); + + stream->next_out += this_len; + stream->avail_out -= this_len; + + job->scoop_avail -= this_len; + job->scoop_next += this_len; + + job->copy_len -= this_len; + + rs_trace("caught up on %ld copied bytes from scoop, %ld remain there, " + "%ld remain to be copied", + (long) this_len, (long) job->scoop_avail, (long) job->copy_len); +} + + + +/** + * Catch up on an outstanding copy command. + * + * Takes data from the scoop, and the input (in that order), and + * writes as much as will fit to the output, up to the limit of the + * outstanding copy. + */ +static void rs_tube_catchup_copy(rs_job_t *job) +{ + rs_buffers_t *stream = job->stream; + + assert(job->write_len == 0); + assert(job->copy_len > 0); + + if (job->scoop_avail && job->copy_len) { + /* there's still some data in the scoop, so we should use that. */ + rs_tube_copy_from_scoop(job); + } + + if (job->copy_len) { + size_t this_copy; + + this_copy = rs_buffers_copy(stream, job->copy_len); + + job->copy_len -= this_copy; + + rs_trace("copied %.0f bytes from input buffer, %.0f remain to be copied", + (double) this_copy, (double) job->copy_len); + } +} + + +/* + * Put whatever will fit from the tube into the output of the stream. + * Return RS_DONE if the tube is now empty and ready to accept another + * command, RS_BLOCKED if there is still stuff waiting to go out. + */ +int rs_tube_catchup(rs_job_t *job) +{ + if (job->write_len) + rs_tube_catchup_write(job); + + if (job->write_len) { + /* there is still write data queued, so we can't send + * anything else. */ + return RS_BLOCKED; + } + + if (job->copy_len) + rs_tube_catchup_copy(job); + + if (job->copy_len) { + if (job->stream->eof_in && !job->stream->avail_in && !job->scoop_avail) { + rs_log(RS_LOG_ERR, + "reached end of file while copying literal data through buffers"); + return RS_INPUT_ENDED; + } + + return RS_BLOCKED; + } + + return RS_DONE; +} + + +/* Check whether there is data in the tube waiting to go out. So if true + * this basically means that the previous command has finished doing all its + * output. */ +int rs_tube_is_idle(rs_job_t const *job) +{ + return job->write_len == 0 && job->copy_len == 0; +} + + +/** + * Queue up a request to copy through \p len bytes from the input to + * the output of the stream. + * + * The data is copied from the scoop (if there is anything there) or + * from the input, on the next call to rs_tube_write(). + * + * We can only accept this request if there is no copy command already + * pending. + */ +/* TODO: Try to do the copy immediately, and return a result. Then, + * people can try to continue if possible. Is this really required? + * Callers can just go out and back in again after flushing the + * tube. */ +void rs_tube_copy(rs_job_t *job, int len) +{ + assert(job->copy_len == 0); + + job->copy_len = len; +} + + + +/* + * Push some data into the tube for storage. The tube's never + * supposed to get very big, so this will just pop loudly if you do + * that. + * + * We can't accept write data if there's already a copy command in the + * tube, because the write data comes out first. + */ +void +rs_tube_write(rs_job_t *job, const void *buf, size_t len) +{ + assert(job->copy_len == 0); + + if (len > sizeof(job->write_buf) - job->write_len) { + rs_fatal("tube popped when trying to write %ld bytes!", + (long) len); + } + + memcpy(job->write_buf + job->write_len, buf, len); + job->write_len += len; +} diff --git a/rsync/types.h b/rsync/types.h new file mode 100644 index 0000000..47ff97c --- a/dev/null +++ b/rsync/types.h @@ -0,0 +1,36 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + + +#if defined(HAVE_STDINT_H) +# include <stdint.h> +#elif SIZEOF_UNSIGNED_INT == 4 +# define uint32_t unsigned int +#elif SIZEOF_UNSIGNED_LONG == 4 +# define uint32_t unsigned long +#elif SIZEOF_UNSIGNED_SHORT == 4 +# define uint32_t unsigned short +#else +# error "can't find an appropriate 32-bit integer type" +#endif + diff --git a/rsync/util.c b/rsync/util.c new file mode 100644 index 0000000..f0f3973 --- a/dev/null +++ b/rsync/util.c @@ -0,0 +1,70 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + + /* + | On heroin, I have all the answers. + */ + + +#include <config_rsync.h> + +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +#include "util.h" +#include "rsync.h" +#include "trace.h" + +void +rs_bzero(void *buf, size_t size) +{ + memset(buf, 0, size); +} + + +void * +rs_alloc_struct0(size_t size, char const *name) +{ + void *p; + + if (!(p = malloc(size))) { + rs_fatal("couldn't allocate instance of %s", name); + } + rs_bzero(p, size); + return p; +} + + + +void * +rs_alloc(size_t size, char const *name) +{ + void *p; + + if (!(p = malloc(size))) { + rs_fatal("couldn't allocate instance of %s", name); + } + + return p; +} diff --git a/rsync/util.h b/rsync/util.h new file mode 100644 index 0000000..2793256 --- a/dev/null +++ b/rsync/util.h @@ -0,0 +1,44 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- library for network deltas + * $Id$ + * + * Copyright (C) 1999, 2000, 2001 by Martin Pool <mbp@samba.org> + * Copyright (C) 1999 by Andrew Tridgell <tridge@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +void * rs_alloc(size_t size, char const *name); +void *rs_alloc_struct0(size_t size, char const *name); + +void rs_bzero(void *buf, size_t size); + + +/* + * Allocate and zero-fill an instance of TYPE. + */ +#define rs_alloc_struct(type) \ + ((type *) rs_alloc_struct0(sizeof(type), #type)) + + +#ifdef __GNUC__ +# define UNUSED(x) x __attribute__((unused)) +#elif __LCLINT__ +# define UNUSED(x) /*@unused@*/ x +#else /* !__GNUC__ && !__LCLINT__ */ +# define UNUSED(x) x +#endif /* !__GNUC__ && !__LCLINT__ */ diff --git a/rsync/version.c b/rsync/version.c new file mode 100644 index 0000000..a76f826 --- a/dev/null +++ b/rsync/version.c @@ -0,0 +1,33 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- dynamic caching and delta update in HTTP + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +#include <config_rsync.h> + +#include <stdio.h> + +#include "rsync.h" + + +/** \brief Library version string. */ +char const rs_librsync_version[] = (PACKAGE " " VERSION); + + diff --git a/rsync/whole.c b/rsync/whole.c new file mode 100644 index 0000000..153d402 --- a/dev/null +++ b/rsync/whole.c @@ -0,0 +1,180 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2000, 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + /* + | Is it possible that software is not + | like anything else, that it is meant + | to be discarded: that the whole point + | is to always see it as a soap bubble? + | -- Alan Perlis + */ + + + +#include <config_rsync.h> + +#include <assert.h> +#include <stdlib.h> +#include <unistd.h> +#include <stdio.h> +#include <string.h> +#include <errno.h> + +#include <rsync.h> + +#include "trace.h" +#include "fileutil.h" +#include "sumset.h" +#include "job.h" +#include "buf.h" +#include "whole.h" +#include "util.h" + +/** + * Run a job continuously, with input to/from the two specified files. + * The job should already be set up, and must be free by the caller + * after return. + * + * Buffers of ::rs_inbuflen and ::rs_outbuflen are allocated for + * temporary storage. + * + * \param in_file Source of input bytes, or NULL if the input buffer + * should not be filled. + * + * \return RS_DONE if the job completed, or otherwise an error result. + */ +rs_result +rs_whole_run(rs_job_t *job, FILE *in_file, FILE *out_file) +{ + rs_buffers_t buf; + rs_result result; + rs_filebuf_t *in_fb = NULL, *out_fb = NULL; + + if (in_file) + in_fb = rs_filebuf_new(in_file, rs_inbuflen); + + if (out_file) + out_fb = rs_filebuf_new(out_file, rs_outbuflen); + + result = rs_job_drive(job, &buf, + in_fb ? rs_infilebuf_fill : NULL, in_fb, + out_fb ? rs_outfilebuf_drain : NULL, out_fb); + + if (in_fb) + rs_filebuf_free(in_fb); + + if (out_fb) + rs_filebuf_free(out_fb); + + return result; +} + + + +/** + * Generate the signature of a basis file, and write it out to + * another. + * + * \param new_block_len block size for signature generation, in bytes + * + * \param strong_len truncated length of strong checksums, in bytes + * + * \sa rs_sig_begin() + */ +rs_result +rs_sig_file(FILE *old_file, FILE *sig_file, size_t new_block_len, + size_t strong_len, rs_stats_t *stats) +{ + rs_job_t *job; + rs_result r; + + job = rs_sig_begin(new_block_len, strong_len); + r = rs_whole_run(job, old_file, sig_file); + if (stats) + memcpy(stats, &job->stats, sizeof *stats); + rs_job_free(job); + + return r; +} + + +/** + * Load signatures from a signature file into memory. Return a + * pointer to the newly allocated structure in SUMSET. + * + * \sa rs_readsig_begin() + */ +rs_result +rs_loadsig_file(FILE *sig_file, rs_signature_t **sumset, rs_stats_t *stats) +{ + rs_job_t *job; + rs_result r; + + job = rs_loadsig_begin(sumset); + r = rs_whole_run(job, sig_file, NULL); + if (stats) + memcpy(stats, &job->stats, sizeof *stats); + rs_job_free(job); + + return r; +} + + + +rs_result +rs_delta_file(rs_signature_t *sig, FILE *new_file, FILE *delta_file, + rs_stats_t *stats) +{ + rs_job_t *job; + rs_result r; + + job = rs_delta_begin(sig); + + r = rs_whole_run(job, new_file, delta_file); + + if (stats) + memcpy(stats, &job->stats, sizeof *stats); + + rs_job_free(job); + + return r; +} + + + +rs_result rs_patch_file(FILE *basis_file, FILE *delta_file, FILE *new_file, + rs_stats_t *stats) +{ + rs_job_t *job; + rs_result r; + + job = rs_patch_begin(rs_file_copy_cb, basis_file); + + r = rs_whole_run(job, delta_file, new_file); + + if (stats) + memcpy(stats, &job->stats, sizeof *stats); + + rs_job_free(job); + + return r; +} diff --git a/rsync/whole.h b/rsync/whole.h new file mode 100644 index 0000000..67e54f7 --- a/dev/null +++ b/rsync/whole.h @@ -0,0 +1,24 @@ +/*= -*- c-basic-offset: 4; indent-tabs-mode: nil; -*- + * + * librsync -- the library for network deltas + * $Id$ + * + * Copyright (C) 2001 by Martin Pool <mbp@samba.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + + +rs_result rs_whole_run(rs_job_t *job, FILE *in_file, FILE *out_file); |