%% HSEIWP.96.10.tex
%% Description:HSEI Working Paper
%% Time-stamp: <Thu Aug 28 12:04:32 CEDT 1997>
%%
%% Preamble
\documentclass[a4paper]{article}
% bibliography style
\usepackage{natbib}
\bibliographystyle{plainnat}

%% add section symbol to headers and starting counting at zero
\setcounter{section}{-1}
% raise the section symbol slightly to match headers
\renewcommand{\thesection}{\S\arabic{section}}
\renewcommand{\thesubsection}{\thesection .\arabic{subsection}}
\renewcommand{\thesubsubsection}{\thesubsection .\arabic{subsubsection}}

%% headers and footers
\usepackage{fancyhdr}
% chapter and section commands use the plain style so this must
% be redefined here
\fancypagestyle{plain}{%
  \fancyhf{}% clear all header/footer fields
  \fancyfoot[C]{\bfseries \thepage}% except the center
  \renewcommand{\headrulewidth}{0pt}
  \renewcommand{\footrulewidth}{0pt}}

% redefine the fancy page style
\fancypagestyle{hseiwp}{%
  \fancyhf{}% clear all header/footer fields
  \fancyhead[R]{\bfseries HSEIWP 96.10}
  \fancyfoot[C]{\bfseries \thepage}}% except the center
\pagestyle{hseiwp}
%% end headers and footers

\title{%
  {\normalsize\textsc{Hebrew Syntax Encoding Initiative Working Paper 96.10}}\\
  \bigskip
  \bigskip
  The Concept of \\the Hebrew Syntax Encoding Initiative}

\author{Kirk E. Lowery}
\date{11 December 1996}

%% End preamble

\begin{document}
\maketitle
\section{Introduction}

The decade of the 1980s saw the development of databases which were used to
assist the linguistic research of biblical scholars. These databases used
various transliteration systems, data structures and programs designed
mostly ad hoc to accomplish specific goals. Since the use of the computer
to assist the research was still in its infancy, this situation was normal
and to be expected.

Problems arose in the course of research. Initially, the computer was used
simply as an automated filing system, an electronic version of a \textsc{3x5}
card-file box. The full potential of the computer as a pattern matcher, as a
tool to simulate theories of language and text had yet to be realized.%
%
\footnote{An outstanding exception to this was Prof. Eep Talstra and the
  \emph{Werkgroep Informatica} at the Vrije Universiteit, Amserdam. Since
  the late 1970s this group has been pursuing the use of computer as an
  analytical tool, and is one of the pioneers of the application of
  computer science to biblical studies.}
%
Even worse, was the assumption that a computer-generated set of numbers,
graphs and text was conclusive, simply because they were
computer-generated. Early uses of the computer included statistical
studies. But these studies did not state the underlying assumptions of
programs, coding of data or of the statistics used.%
%
\footnote{For a devastating critique of the use of statistics in biblical
  studies, see \cite{Forbes1992}.}
%
There was no way to reproduce the results independently, algorithms were
not published so that peer review could reveal errors, the data files were
unavailable and unexamined. Hence there was no way that the results could
be repeated and confirmed by the academic community.

For the Hebrew Bible, this situation began to improve with the release of
the first version of the \emph{Westminster Morphologically Tagged Hebrew
Bible}~\citep{Groves1994}.  At last, there was a public standard of
morphological parsing, which could be used and checked. More importantly,
that data could now be constantly improved with use by the general public.

The next logical step was for this linguistic analysis of the Hebrew Bible
be extended to the syntactic level. However, the problem is now
qualitatively different. Unlike morphology, there is no generally accepted
set of syntactic features which should be encoded, nor is there a consensus
on the meaning of those features. This reflects the conditions of the field
of general linguistics. Syntactic theory has been in flux for a number of
years, especially with the rise of discourse analysis (text linguistics).
Concerns for the ``grammar'' of the text brought into question traditional
syntactic descriptions, and also brought the additional concerns of the
semantics of syntax. Different theoretical perspectives had been evolving
for the previous forty years.%
%
\footnote{For a review of these developments see \cite{Moravcsik1980},
    \cite{Sells1985}, and \cite{Pollard1994}. Recently an attempt is being
    made to explain syntax only in terms of morphology and lexicon
    \citep{Chomsky1995}.}
%

\section{ The Goals of HSEI}

HSEI is first and foremost a research project investigating the question,
\begin{center}
\emph{How can the syntax of Biblical Hebrew be represented in a database so 
that
the full potential of the computer as a research tool can be utilized?}
\end{center}
Along the way toward answering this question, the concrete tasks are:

\begin{itemize}
\item to create a ``theory-neutral'' set of tags for encoding syntactic
  notions
\item to create a publicly available tagged Hebrew Bible database
\item to provide a set of rudimentary software tools to manipulate the
  database
\end{itemize}

There is no question that ``theory neutral'' is a difficult---if not
impossible---goal. Perhaps a better term might be ``theory configurable.''
The database and its tagging scheme must be easily adapted in arbitrary
ways. By the very nature of research, there is no way to predict how
researchers in the future might want to use and modify a syntactic
database. How can we design the data structures to allow for this?

\subsection{Algorithmic Tagging}

Computers have the attribute of following instructions literally and
mindlessly. They can be absolutely consistent in how they act. By
submitting a set of rules to a \textsc{parser},%
%
\footnote{A program which accepts a set of symbols and a set of rules to
  act on those symbols in a consistent manner.}
%
the program could tag the text in a consistent manner. Since our ability to
tag the text via rules is incomplete, and because natural language is
complex and inconsistent as compared to artificial languages such as those
used for programming, a \textsc{grammar} for the parser would be
insufficient to handle all cases in the text. Hence we also provide an
\textsc{exception list} which instructs the parser to act contrary to its
grammar at certain points. The exception list then becomes the raw material
for further analysis and refinement of the algorithms and rules used in the
tagging.

With a beginning input file from MORPH, and armed with the parser grammar
and exception list, the researcher can reproduce precisely the same tagged
file as anyone else using that same set of rules and exceptions. At the
same time, the researcher is free to modify the rules and exceptions as
desired to produce different results.

This approach gives a great deal of freedom to the researcher, but not
complete freedom. There is still the matter of the number and kind of tags
which are used, and even the amount of information available to the parser
in the first place. HSEI plans to use MORPH as the basic input, but this
may be unacceptable to someone in the future. The format, ordering and
connections that MORPH supplies could conceivably be unacceptable or
inappropriate for the questions that are being asked of it. How can we
allow the user to ``reconfigure'' the tags, their use and meaning as well
as the type of data which is input?


\subsection{The Use of SGML}

There is a system of encoding the information structure of a file in a
completely independent way, one which is also in widespread use. It is
called ``Standard Generalized Markup Language'' or SGML.%
%
\footnote{The reader may be familiar with the more common
  \textsc{HTML}---ubiquitous on the Internet---is a subset of SGML.}
%
SGML itself is not a scheme for marking up, but is a ``meta-language'' for
describing markup schemes, which is what is meant by ``Generalized.'' SGML
is an ISO standard (8879)%
%
\footnote{There is no better beginning place for information about SGML than
  Robin Cover's SGML Home Page, \texttt{<http://www.sil.org/sgml/sgml.html>}.}
%
 which defines how the information structure of a
document may be tagged. This ``Document Type Definition'' provides the
rules for how and when tags are introduced.%
%
\footnote{It is important for the reader to understand that while visual
  formatting can be done using SGML, this is only a small part of what SGML
  was designed to do.}
%
SGML is about \textsc{information markup}, of which visual formatting is
only one part. Once the DTD has been written, the text can be marked up
according to the rules provided in the DTD. Then an SGML parser---which
understands DTDs---can be used to (1) validate that the markup was
correctly done according to the DTD and (2) provide an output suitable for
input into another program for further processing.

If a DTD could be written for the information structure of a Hebrew syntax
database, and if the grammar for the tagging parser followed that DTD, then
the researcher could now access the information in the database at all
levels.  Because this layering of coding, the user change choose just how
deeply it is necessary to go in order to produce the desired results.

Writing a DTD for complex information structures can be a daunting task.
Fortunately, there is a major effort underway to apply the full power of
SGML to the humanities. The Text Encoding Initiative (TEI) is an
international consortium which has produced DTDs for a wide range of
documents, including the markup of linguistic analysis.%
%
\footnote{The TEI Home Page is 
\texttt{<http://www.uic.edu/orgs/tei/index.html/>}.}
%
The sophistication of the TEI DTD for linguistic analysis is not yet at a
level suitable for direct adoption by HSEI. In fact, this is an area where
HSEI hopes to make a contribution to the larger question of linguistic
markup.

When completed, HSEI hopes to provide:

\begin{itemize}
\item the community of biblical scholars with a public baseline against
  which all syntactic research can be replicated and confirmed by peer
  review (something which does not exist at present, although private and
  proprietary databases of the Hebrew Bible are being developed and used)
\item the software developer with the raw material for a new generation of
  Bible study software which allows the non-specialist access to the syntax
  of original language
\item the humanities with experience, an example, algorithms and
  applications of how the computer can and ought to be used in the study
  and analysis of texts
\item the linguist interested in text linguistics (discourse analysis) with
  a tool which can be used to investigate the discourse grammatical
  features of the Hebrew Bible
\end{itemize}

\section{Great Expectations}

HSEI is an open project, with working papers, preliminary versions of
encoded data, software and bibliography available via a web page. HSEI
welcomes and invites public scrutiny of its ongoing research, as well as
participation by qualified persons. While HSEI will retain copyright of
official publications and the final database of encoded text, the database
will remain available via \texttt{ftp} to be used freely. Software
developed by the project will be copyrighted under the Free Software
Foundation's General Public License. Apart from this, the work done by
individual members of HSEI will remain their own.

Almost every aspect of HSEI is experimental. Even the present conception of
the project is vastly different from the original forecast five years ago,
when we first began discussing the matter after the completion of the first
version of MORPH. The working papers for HSEI are the diaries of the
member's efforts, and will reveal our advances, our mistakes, our
increasing understanding, and our appalling ignorance.

We stand on the shoulders of those whose open-handed generosity has gone
before us. We desire to emulate that same tradition of scholarship and
offer this to the public in the hopes that others will be encouraged and
enabled in some small way in their own efforts.

\bibliography{basicHSEI}

\end{document}