How to talk about information: Some simple ways
This section begins with a quote from Strawson's notable 1950 symposium with Austin, which centers on the concept of truth Austin's initial perspective in this discussion serves as an appropriate foundation for our exploration of information.
Austin sought to clarify the concept of truth, making it more accessible for discussion by highlighting that 'truth' is an abstract noun, similar to 'information.' This distinction will be a key focus in the initial section of this thesis.
“ ‘What is truth?’ said jesting Pilate, and would not stay for an answer.” Said Austin: “Pilate was in advance of his time.”
As with truth, so with 1 information:
For ‘truth’[‘information’]itself is an abstract noun, a camel, that is of a logical construction, which cannot get past the eye even of a grammarian.
We examine the nature of Truth, questioning whether it is a substance, akin to a Body of Knowledge, a quality that exists within messages, or a relational concept defined by correspondence or correlation.
But philosophers should take something more nearly their own size to strain at What needs discussing rather is the use, or certain uses, of the word
A characteristic feature of abstract nouns is that they do not serve to denote kinds of entities having a location in space and time An abstract noun may be either a count
Chapter 1 explores the concept of information, defining it as an abstract mass noun that contrasts with concrete mass nouns like "water" and abstract count nouns such as "number." Abstract nouns often derive from adjectives or verbs for grammatical convenience, as seen in the transition from "truth" to "true." Similarly, "information" is rooted in the verb "inform," which means to bring someone to know something they previously did not know Understanding this relationship clarifies the essence of information without relying on circular definitions.
This article will concentrate on the fundamental aspects of the concept of information, rather than providing an exhaustive examination of its various definitions or delving into the philosophical nuances associated with the term.
A key distinction exists between the everyday understanding of information and the technical definitions established by Shannon in 1948 While the everyday concept links closely to knowledge, language, and meaning, it fundamentally relies on the presence of a person or language user who can read, understand, and utilize the information, as well as encode or decode it.
A technical definition of information relies solely on mathematical and physical terminology, resulting in minimal direct connections to semantic and epistemic concepts.
A technical notion of information might be concerned with describing correlations and the statistical features of signals, as in communication theory with the Shan-
2 An illuminating discussion of mass, count and abstract nouns may be found in Rundle (1979, §§27-29).
3 For discussion of Dretske’s opposing view, however, see below, Section 1.5.
Chapter 1 explores various concepts of information, highlighting its connection to statistical inference as noted by researchers like Fisher (1925) and Kullback and Leibler (1951) It introduces a technical understanding of information that encompasses abstract ideas of structure, including complexity as discussed in algorithmic information theory by Chaitin (1966), Kolmogorov (1965), and Solomonoff (1964) Additionally, it considers the functional role of information, particularly in biological contexts, referencing Jablonka (2002).
This thesis focuses on information theory, encompassing both quantum and classical aspects We will primarily examine the well-established concept of Shannon information, while also addressing related concepts from both classical and quantum information theory to provide a comprehensive understanding.
The term "information" is commonly used in phrases like "information about p" or "information that q," indicating its intentionality—meaning it is directed towards specific objects, events, or topics This intentionality reflects a focus on something that may or may not be physically present, highlighting a characteristic that is challenging to categorize within the purely physical realm.
Information is closely tied to the concept of knowledge, where we can differentiate between possessing information, acquiring it, and containing it Possessing information equates to having knowledge, while acquiring information involves gaining knowledge through various means such as asking, reading, or perceiving When something is said to contain information, it implies that it can provide or contribute to knowledge There are at least two distinct ways to understand this relationship between information and knowledge.
4 N.B To my mind, however, Jablonka overstates the analogies between the technical notion she introduces and the everyday concept.
Communication theory often shares similar concepts and mathematical expressions with statistical inference Notably, there are connections between algorithmic information and Shannon information, as demonstrated by Bennett (1982), which shows that the average algorithmic entropy of a thermodynamic ensemble equals the Shannon entropy of that ensemble.
6 Containing information and containing knowledge are not always the same: we might, for example say that a train timetable contains information, but not knowledge.
CHAPTER 1 CONCEPTS OF INFORMATION 6 in which this may be so.
It is primarily a person of whom it can be said that they possess information, whilst it is objects like books, filing cabinets and computers that contain information (cf Hacker,
1987) In the sense in which my books contain information and knowledge, I do not.
Information storage involves organizing data in a way that allows it to be easily accessed and understood In the context of computers, this data is encoded so that facts and figures can be decoded and read when needed.
On a plausible account of the nature of knowledge originating with Wittgenstein (e.g Wittgenstein, 1953, §150) and Ryle (1949), and developed, for example by White
According to scholars like Kenny (1989) and Hyman (1999), knowledge is defined as a specific capacity or ability, rather than merely a state of being This distinction highlights that possessing information equates to having a certain skill set, while containing information refers to holding specific categorical properties at a given moment Further exploration of this distinction is available in the works of Kenny (1989) and Timpson (2000).
Information can be contained in various forms, with a key distinction between propositional and inferential information Propositional information is closely linked to the expression of specific propositions, which can be found in physical formats such as books or filing cabinets, or in digital formats like computers and removable disks These objects, whether tangible or digital, serve as the carriers of the information contained within them.
The Shannon Information and related concepts
Interpretation of the Shannon Information
It is instructive to begin by quoting Shannon:
The core challenge of communication lies in accurately conveying a message from one location to another While these messages often carry meaning, the semantic elements are not essential to the engineering aspects of the communication process.
The communication system comprises an information source, a transmitter (or encoder), a potentially noisy channel, and a receiver (decoder) It is designed to handle any message produced, regardless of its meaning or relevance to the recipient Notably, Shannon's theory suggests that in many practical applications, the transmitted messages often lack inherent meaning For instance, in a telephone conversation, the transmission consists of an analog signal that captures the speaker's sound waves, which are then digitally encoded for transmission.
In Shannon's theory, 'information' pertains to the characteristics of the message source rather than individual messages This characterization aims to determine the necessary capacity of a communications channel to effectively transmit all messages generated by the source The concept of Shannon information is introduced to facilitate this understanding, emphasizing that the statistical properties of the source can help minimize the channel capacity required for message transmission, focusing specifically on discrete messages for simplicity.
In an ensemble X of letters {x1, x2, , xn} with associated probabilities p(xi), we derive messages consisting of N letters from this source For large values of N, typical sequences of letters will reflect the probabilities, containing approximately N p(xi) occurrences of each letter xi Consequently, the number of distinct typical sequences of letters can be determined based on these probabilities.
10 More properly, this ensemble models the source.
CHAPTER 1 CONCEPTS OF INFORMATION 12 and using Stirling’s approximation, this becomes 2 N H(X) , where
X i=1 p(xi) logp(xi), (1.1) is the Shannon information (logarithms are to base 2 to fix the units of information as binary bits).
As N approaches infinity, the likelihood of encountering an atypical sequence diminishes, leaving us with only 2^(N H(X)) equiprobable typical sequences to consider as potential messages Consequently, we can encode each typical sequence using a binary code consisting of N H(X) bits, allowing us to transmit this compact representation instead of the original message comprised of N letters, which would require N log n bits.
Shannon's noiseless coding theorem demonstrates that a message can be compressed from N letters to N H(X) bits, achieving optimal compression, as outlined in his 1948 work This concept establishes Shannon information as a measure of information, indicating the maximum compression possible for messages formed from an ensemble X Additionally, it can be stated that the information per letter in a message is H(X) bits, reflecting the source's information However, it is crucial to note that this consideration applies specifically to letters xi drawn from the ensemble.
X to have associated with it the informationH(X) if we consider it to be a member of a typical sequence of N letters, whereN is large, drawn from the source.
It is crucial to avoid the misconception that Shannon information, which indicates the maximum compressibility of a message from a set, also defines the message's irreducible meaning in bits This misunderstanding arises from not differentiating between a code, which is indifferent to meaning, and a language, which inherently involves meaning (Harris, 1987).
CHAPTER 1 CONCEPTS OF INFORMATION 13 Information and Uncertainty
The Shannon information quantifies the expected information gain from conducting a probabilistic experiment, serving as both a measure of uncertainty and information It reflects the lack of concentration in a probability distribution, indicating our uncertainty regarding potential outcomes Uffink (1990) offers an axiomatic characterization of uncertainty measures, establishing a general class, Ur(~p), that includes the Shannon information among its members.
1989) The key property possessed by these measures is Schur concavity (for details of the property of Schur concavity, see Uffink (1990), Nielsen (2001) and Section 2.3.1 below).
Imagine a random probabilistic experiment described by a probability distribution
The relationship between uncertainty and information is clear: higher uncertainty in a distribution, represented as p={p(x1), , p(xn)}, indicates a greater potential for gaining knowledge from the results of an experiment This concept is further refined through Shannon information, which quantifies the value of the information gained from learning the outcome.
When evaluating the outcomes of a probabilistic experiment, it's important to understand that the shape of the probability distribution does not necessarily indicate the specific result of an individual trial, as any outcome with a non-zero probability can occur However, the probability distribution allows us to assign value to each potential outcome Likely outcomes carry less value since their occurrence is expected, while unlikely outcomes are surprising and thus possess greater value.
The function -logp(xi) represents the 'surprise' information linked to the outcome xi, indicating how unexpected the observation of this outcome is based on its probability This measure highlights the significance of observing the outcome in the context of the known probability distribution, emphasizing the value of the information gained from the experiment.
11 Of course, this is a highly restricted sense of ‘value’ It does not, for example, refer to how much
The expected information gain from an outcome, represented as −logp(xi), can be quantified using the expectation value of the surprise information, P ip(xi)(−logp(xi)) This relationship illustrates that the Shannon information, H, of a probability distribution ~p reflects our anticipated information gain prior to conducting an experiment Thus, Shannon information serves as a measure of the expected information we can acquire.
Measures of uncertainty, such as Ur(~p), can be interpreted as indicators of information gain Conversely, measures of knowledge associated with a probability distribution serve as inverses of uncertainty; a more concentrated probability distribution indicates greater knowledge about potential outcomes, enhancing our predictive capabilities This perspective emphasizes that possessing information about an experiment's outcome does not imply having partial knowledge of a predetermined fact, but rather reflects our understanding of the likelihood of various results.
The minimum number of questions needed to specify a sequence
The Shannon information is commonly understood as the minimum average number of binary questions required to identify a sequence from a given ensemble This interpretation, however, does not seem to offer a perspective on Shannon information that is entirely separate from earlier concepts.
Imagine that a long sequence N of letters is drawn from the ensemble X, or that
In independent experiments with outcomes that have specific probabilities p(xi), the actual list of outcomes remains concealed Our objective is to uncover this sequence by posing questions that can only be answered by the guardian of the sequence.
More on communication channels
In our exploration of communication systems, we have primarily focused on defining the information source However, it is equally essential to analyze and characterize the communication channel to gain a comprehensive understanding of the entire system.
A channel is a device that maps a set of input states {xi} to a set of output states {yj} In the case of a noisy channel, this mapping is not one-to-one, meaning a single input can lead to multiple possible outputs due to noise interference The fundamental type of channel, known as the discrete memoryless channel, is defined by the conditional probabilities p(yj|xi), which indicate the likelihood of producing output yj when input xi is applied.
When the probability distribution p(xi) for various input states is defined, we can compute the joint distribution p(xi∧yj) In this context, we treat the input state prepared during a channel use as a random variable X, where p(X = xi) = p(xi) Similarly, the output generated is represented as a random variable Y, with p(Y = yj) = p(yj) Additionally, we can define the joint random variable X∧Y, where p(X∧Y = xi∧yj) = p(xi∧yj).
The joint distributionp(xi∧yj) allows us to define the joint uncertainty
H(X∧Y) =−X i,j p(xi∧yj) logp(xi∧yj), (1.2) and an important quantity known as the ‘conditional entropy’:
H(X|Y) =X j p(yj) −X i p(xi|yj) logp(xi|yj)
The term "scare quotes" highlights that this quantity does not represent entropy or uncertainty in a traditional sense; instead, it reflects the average uncertainties of the conditional distributions for the input based on a specific output, Y This metric assesses the average level of uncertainty regarding the X value once an output Y has been observed.
As Uffink (1990, §1.6.6) notes, it pays to attend to the fact that H(X|Y) is not a
CHAPTER 1 CONCEPTS OF INFORMATION 17 measure of uncertainty It is easy to show (e.g Ash, 1965, Thm.1.4.3–5) that
The Shannon measure of information highlights that the uncertainty of a random variable X, given another variable Y, is always less than or equal to the uncertainty of X alone, expressed as H(X|Y) ≤ H(X) This relationship holds true with equality only when X and Y are independent The principle suggests that learning the value of Y provides insight into X, thereby reducing our uncertainty about X, except in cases where the two variables are independent.
The uncertainty of X is never increased by knowledge of Y It will be decreased unless Y and X are independent events, in which case it is not changed (Shannon, 1948, p.53)
The common belief that increased knowledge always reduces uncertainty is misleading As Uffink points out, uncertainty can actually rise after an observation For instance, consider the example of keys: while it may be highly probable that they are in your pocket, the alternative possibilities are numerous and equally likely, resulting in low uncertainty However, if you check and discover the keys are not in your pocket, your uncertainty about their location significantly increases This illustrates that gaining knowledge can sometimes lead to greater uncertainty.
This does not conflict with the inequality (1.4), of course, as the latter involves an average over post-observation uncertainties Uffink remarks, against Jaynes (1957, p.186) for example, that
An increase in uncertainty regarding an experiment's outcome can occur even when additional information about its distribution is provided This confusion arises from the ambiguous use of the term "information" and the misleading label of "conditional entropy," which actually represents an average of the entropies of conditional distributions.
The significance of conditional entropy becomes evident when analyzing a large number \( N \) of repeated uses of a channel In this scenario, there are \( 2^{N H(X)} \) typical input sequences \( X \) that can occur, \( 2^{N H(Y)} \) typical output sequences \( Y \) that can be generated, and \( 2^{N H(X∧Y)} \) typical joint sequences.
CHAPTER 1 CONCEPTS OF INFORMATION 18 sequences of pairs of X, Y values that could obtain Suppose someone observes which
The Y sequence has been generated, but in the presence of noise, multiple input X sequences may lead to the same output Conditional entropy quantifies the potential input sequences that could result in the observed output, considering only those with a non-zero probability.
If there are 2 N H(X∧Y ) typical sequences of pairs ofX, Y values, then the number of typical X sequences that could result in the production of a givenY sequence will be given by
The logarithmic relationship in information theory, expressed as H(X∧Y) = H(Y) + H(X|Y), indicates that the number of input sequences corresponding to a specific output sequence is given by 2^(N H(X|Y)) Shannon (1948) highlights that in the context of a noisy channel, the conditional entropy quantifies the number of bits per letter required to be sent through an auxiliary noiseless channel to correct errors introduced by noise during transmission In scenarios where input and output states are perfectly correlated, meaning there is no noise, the need for error correction diminishes significantly.
Another most important quantity is themutual information,H(X :Y), defined as
It follows from Shannon’s noisy coding theorem (1948) that the mutual information
H(X:Y) governs the rate at which information may be sent over a channel with input distributionp(xi), with vanishingly small probability of error.
In a noiseless channel, the output Y sequence retains the same amount of information as the input X sequence, quantified as N H(X) bits However, in the presence of noise, the information content of the output diminishes The term H(X|Y) indicates the number of bits required to correct the observed Y sequence, leading to the conclusion that the actual information contained in the sequence is represented by N H(X) − N H(X|Y) = N H(X:Y) bits.
The measure N H(X : Y) quantifies the extent to which we can discern the identity of an input sequence X based on the observed output sequence Y Specifically, there are 2 N H(X|Y) input sequences that align with a given output sequence, and the proportion of this subset relative to the total number of possible input sequences indicates how much we have refined our understanding of the X sequence through the observation of the Y sequence.
2 N H(X:Y ) , and the smaller this fraction—hence the greaterH(X :Y)—the more one learns from learning theY sequence.
The noisy coding theorem is crucial for understanding mutual information, especially in the context of large sequences of length N It states that for a given input distribution p(xi), it is possible to identify 2^(N H(X:Y)) code words of length N By observing the Y sequence generated from one of these code words, we can accurately infer the corresponding X sequence, achieving a probability of error that approaches zero as N increases (Shannon, 1948) Furthermore, if we consider an information source W that produces messages with an information content of H(W) = H(X:Y), each output sequence of length N can be linked to an X code word, allowing messages from W to be transmitted over the channel with negligible error as N grows larger.
The capacity, C, of a channel is the maximum achievable rate of information transfer, determined by the supremum of H(X : Y) across all input distributions p(xi) According to the noiseless coding theorem, when a channel has a capacity C and an information source with entropy H that is less than or equal to C, it is possible to develop a coding system that allows the source's output to be transmitted over the channel with an arbitrarily low error rate.
The surprising finding reveals that, despite the presence of noise, it is possible to achieve excellent information transmission without the per letter rate of transmission diminishing to zero This is supported by the noisy coding theorem, which guarantees that efficient communication can still be accomplished under these conditions.
Interlude: Abstract/concrete; technical, everyday
In this chapter, I aim to shift the focus from the challenging question of "What is information?" by following the insights of Austin and Wittgenstein Instead of trying to define 'information' as an abstract noun through elusive references, we should examine straightforward examples of its function This approach allows us to better understand 'information' in relation to simpler, less complex terms such as 'inform.'
Now, when turning to information in the technical sense of Shannon’s theory, we explicitly do not seek to understand this noun by comparison with the verb ‘inform’.
The term 'information' is not a nominalization of a verb and is classified as an abstract noun, meaning it does not refer to a tangible object or substance It's important to recognize that the distinction between abstract and concrete nouns does not necessarily align with the difference between physical concepts and those from other categories.
In Shannon's theory, 'information' is a technical concept that, while definable in physical terms, does not represent a specific entity or substance Similar to energy, which is a fundamental physical concept and an abstract mass noun, information also functions as a property name The distinctions between energy and the technical notion of information as physical quantities warrant deeper exploration, as discussed in Chapter 3, Sections 3.4 and 3.6.
The term ‘information’ in a technical context is an abstract noun, as illustrated by two distinct strategies for defining it within information theory The first approach quantifies information through Shannon information and mutual information, while the second focuses on what is transmitted by information sources These varying strategies highlight the complexity of understanding the concept of information.
Wittgenstein highlights the philosophical confusion that arises from fundamental questions such as "What is length?" and "What is meaning?" These inquiries create a mental struggle, as we feel compelled to identify something tangible in response, yet find it challenging to do so This reflects a key source of bewilderment in philosophy, where the quest for a corresponding object to abstract concepts leads to confusion.
CHAPTER 1 CONCEPTS OF INFORMATION 21 provide differing, but complementary answers Under both, however, ‘information’ is an abstract noun.
The first strategy involves understanding Shannon information and mutual information Shannon information quantifies the compressibility of messages from a source, while mutual information measures a channel's capacity to transmit messages based on a specific input source distribution However, these concepts do not quantify tangible entities; the compressibility of messages and the capacity of a channel are abstract concepts, much like the size of a shoe is not a concrete object.
The second strategy in communication theory focuses on the aim of effectively reproducing a message from one point to another, as highlighted by Shannon In this context, information refers to the data that a communication protocol is designed to transmit, which is generated by an information source Success in transmission depends on accurately reproducing this information However, it is essential to understand that "what is produced" refers to the type or structure of the information rather than the specific instances created by the source at any given time Consequently, "information" in this technical sense remains an abstract concept rather than a tangible entity.
So, for example, if the sourceX produces a string of letters like the following: x2x1x3x1x4 x2x1x7x1x4, say, then the type is the sequence ‘x2x1x3x1x4 x2x1x7x1x4’; we might name this
‘sequence 17’ The aim is to produce at the receiving end of the communication chan- nel another token of this type What has been transmitted, though, the information
The definition of successful transmission in communication is intentionally broad, as it varies based on individual goals and interests when establishing a communication protocol.
CHAPTER 1 CONCEPTS OF INFORMATION 22 transmitted on this run of the protocol, is sequence 17; and this is not a concrete thing.
In both everyday and technical contexts, it is essential to differentiate between the physical representation of information and the underlying propositions they convey For instance, when I write a message to a friend, the actual words on the paper are distinct from the meaning they express Similarly, in technical information transfer, one must distinguish between the concrete outputs produced by a source and the abstract types these outputs represent Ultimately, it is the conveyed meaning, whether in everyday communication or technical transmission, that constitutes the true information being shared.
A significant distinction exists between technical and everyday notions of information In everyday communication, when I write a message to a friend, I convey not just the sentence structures but also the underlying propositions that represent the intended information In contrast, information theory focuses solely on the tokens produced by a source and their types, without considering the meanings or implications of these types Thus, the technical definition of information lacks a semantic dimension, as it does not address what the types signify or how they might be interpreted From the perspective of information theory, the output of an information source is devoid of any syntactic structure.
Aspects of Quantum Information
Quantum information theory explores the unique capabilities of quantum systems for processing and transmitting information This field unites researchers through its emphasis on the distinct advantages offered by quantum mechanics, which enhance communication and computational efficiency beyond classical methods.
15 Note, of course, that the propositions expressed are not to be identified with the sentence types of which the tokens I write are particular instances (Consider, for example, indexicals.)
Chapter 1 discusses the concepts of information, highlighting that the unique properties of quantum systems, such as entanglement and non-commutativity, offer innovative opportunities for communication protocols and information processing As miniaturization trends continue, understanding these quantum features is crucial for overcoming potential nuisances in computation and information transmission.
Deutsch (1985) introduced the concept of the universal quantum computer, highlighting its potential to outperform classical computational models, particularly in factoring large numbers (Shor, 1994) Additionally, quantum cryptography leverages the principle that non-orthogonal quantum states cannot be perfectly distinguished, enabling the secure sharing of random keys (Bennett and Brassard, 1984) Furthermore, entanglement plays a crucial role in enhancing the security of these communication protocols (Ekert, 1991).
The field of quantum information began to take shape in the mid-1980s, but the true understanding of quantum information emerged with the development of the quantum noiseless coding theorem by Schumacher in 1995, building on earlier work by Jozsa and Schumacher in 1994.
Quantum information theory extends classical information theory by introducing new communication elements like qubits and shared entanglement, while also offering quantum adaptations of sources, channels, and codes This article will highlight key results relevant to the field, drawing from foundational works by Nielsen and Chuang (2000), Bouwmeester et al (2000), Preskill (1998), and Bennett and Shor (1998), as well as contributions from Ekert and Jozsa.
(1996) also provides a nice review of quantum computation up to and including the development of Shor’s algorithm.
The first type of task one might consider consists of using quantum systems to
In October, during the IEEE meeting on the Physics of Computation held in Dallas, Ben Schumacher first introduced the concept of quantum information, as noted by Chris Fuchs.
1992 The germ of the idea and the term ‘qubit’ arose in conversation between Schumacher and Wootters some months earlier.
In the realm of quantum systems, while we typically consider an n-dimensional system to have up to n mutually distinguishable states, it can also exist in various non-orthogonal states This flexibility in state preparation comes at the cost of not being able to perfectly identify the prepared state Consequently, this leads to an important distinction not present in classical information theory: the differentiation between specification information, which is necessary to define the state of a quantum system, and accessible information, which is the actual information encoded within the system.
In this article, we explore the encoding of outputs from a classical information source, A, which produces outputs \( a_i \) with associated probabilities \( p(a_i) \) Upon receiving an output \( a_i \), a quantum system is prepared in the corresponding signal state \( \rho_{a_i} \) These signal states can be either pure or mixed and may or may not be orthogonal However, when the number of outputs \( k \) exceeds the dimension \( n \) of the quantum systems, the signal states must necessarily be non-orthogonal.
In the context of preparing sequences of signal states of length N, where N is significantly large, the information required to define this sequence amounts to N H(A) bits Consequently, the specification information represents the number of bits per system necessary to describe the entire sequence of states, which is derived from the information of the classical source.
The quantum analogue of the Shannon informationH is thevon Neumann entropy (Wehrl, 1978):
In the context of quantum mechanics, the density operator ρ represents a system in an n-dimensional Hilbert space, with eigenvalues denoted as λi As the number of quantum systems, N, becomes very large, the collection of these systems can be effectively described as an ensemble characterized by the density operator ρk, leading to the expression X i=1 λilogλi.
Equally, if one does not know the output of the classical source on a given run of the
CHAPTER 1 CONCEPTS OF INFORMATION 25 preparation procedure, then the state of the individual system prepared on that run may also be described by this density operator.
The von Neumann entropy takes its maximum value, logn, when ρ is maximally mixed, and its minimum value, zero, ifρis pure It also satisfies the inequality (Wehrl, 1978):
The equation X i p(ai)S(ρa i) holds true if and only if the states ρa i are mutually orthogonal, meaning ρa iρa j = 0 for i ≠ j Consequently, the informational capacity of the sequence, constrained only by the number of outputs k from the classical source, can significantly exceed its von Neumann entropy, which is restricted by the dimensionality of the quantum systems involved.
To determine how much information we can encode in quantum systems, we must examine the measurements made on these systems and the resulting mutual information H(A:B) Here, B represents the observable being measured, with outcomes bj and probabilities p(bj) For successful encoding, which implies the possibility of decoding, the maximum information encoded is defined by the accessible information, as outlined by Schumacher (1995) A significant finding by Holevo (1973) establishes an upper limit on the mutual information derived from measuring any observable, including positive operator valued (POV) measurements, which can have more outcomes than the system's dimensionality.
X i p(ai)S(ρa i), (1.9) with equalityiff theρa i commute.
The Holevo bound (1.9) implies the weaker inequality
Chapter 1 discusses the fundamental concepts of information, emphasizing that the maximum information encoded in a quantum system is constrained by the number of orthogonal states, which corresponds to the dimension of the system's Hilbert space Even with POV measurements aimed at distinguishing non-orthogonal states, a single qubit can encode a maximum of one bit of information This limitation is further illustrated by the Holevo bound and inequality, underscoring the inherent restrictions in quantum information theory.
When encoding states ρa i are not orthogonal, the resulting inequality becomes strict, indicating that the accessible information is less than the specified information H(A) This highlights the challenge of determining the prepared states when using non-orthogonal encodings Additionally, if the conditional information H(A :B) is less than H(A) for any measurement B, it becomes impossible to accurately identify the sequence of prepared states through measurements.
Quantum coding can be approached by starting with a quantum source instead of a classical one In this context, a classical source is represented by an ensemble A, where letters ai are drawn with specific probabilities p(ai) Similarly, a quantum source is modeled by an ensemble of systems in pure states ρai, produced with the same probabilities p(ai) (Schumacher, 1995) This leads to the introduction of the qubit as a measure of quantum information, analogous to how Shannon’s noiseless coding theorem defines the bit for classical information, thereby characterizing the quantum source effectively.
Information is Physical: The Dilemma
A prominent assertion in quantum information theory and quantum computation is that "Information is Physical." This statement, while intriguing, raises conceptual questions about its true meaning and implications Despite its ambiguity, the phrase is frequently referenced in discussions within the field.
The question of cloning in quantum mechanics emerged within the framework of state determination and superluminal signaling If cloning an unknown quantum state were feasible, it would allow for the duplication of a system into multiple copies, enabling the identification of that state This capability could lead to superluminal signaling through entanglement in an EPR scenario, as it would facilitate the differentiation between various preparations of the same density matrix, thus revealing which measurement was conducted on a remote half of an EPR pair.
Considering only unitary evolutions may seem restrictive; however, non-unitary processes, such as measurements, can be viewed as unitary evolutions within a larger framework By incorporating auxiliary systems, including the state of the measuring apparatus, the fundamental argument remains intact.
Chapter 1 explores the core concepts of information, highlighting its essential role in quantum information theory It emphasizes that advancements in both theoretical and practical aspects of quantum information and computation often imply or suggest these foundational insights.
The slogan "Information is Physical" presents a significant dilemma, as it suggests a reductionist view that central semantic and mental attributes can be reduced to physical concepts This philosophical assertion is contentious and challenging to substantiate, particularly in light of the claims and successes of quantum information theory in physics.
The term 'information' in the slogan may be interpreted in a technical context, suggesting that a defined physical quantity is indeed physical However, this assertion lacks significant impact, as it does not provide a groundbreaking theoretical insight.
In philosophy, there exists a tradition known as semantic naturalism, where proponents aim to reduce semantic concepts to physical ones, exemplified by thinkers like Dretske (1981) Semantic naturalizers would likely agree that everyday information is physical, but this perspective does not influence the philosophical implications of the statement "Information is Physical." This phrase addresses the relationship between different concepts rather than engaging with quantum information theory, which focuses on how quantum systems, with their unique properties, can facilitate information processing and transmission Consequently, quantum information theory does not provide evidence for or against the philosophical reduction of semantic properties to physical ones, and this limitation does not detract from its value.
21 Perhaps the most vociferous proponent of the idea that information is physical was the late Rolf Landauer (e.g Landauer, 1991, 1996).
22 Another possible reading of the slogan will be discussed briefly at the end of Chapter 6.
The success of naturalizing semantic properties remains an unresolved issue, with no definitive completion of the project achieved Although Adams (2003) offers an optimistic perspective on progress, recent reviews by Loewer (1997) and McLaughlin and Rey (1998) indicate that significant systematic challenges still persist within the project.
As noted in these reviews, proposals for naturalizing semantics typically face two sorts of problems, whose ancestry, in fact, may be traced back to difficulties that Grice
(1957) raised for the crude causal theory of meaning These are what may be called the problem of error and theproblem of fine grain.
A naturalized account of semantics must address the content of beliefs, particularly the issue of error, where individuals can hold beliefs that do not align with reality This presents a challenge, as it suggests that belief can only exist when its content is true, which is not the case Additionally, the problem of fine grain arises in defining the intricate structure of beliefs without relying on linguistic elements, since semantic relations are more nuanced than causal ones For instance, the belief that a creature has a heart differs from the belief that it has a kidney, despite both properties being present simultaneously There remains a lack of consensus on whether these challenges can be effectively resolved while still adhering to a naturalistic framework.
Many argue that our inability to offer a satisfactory naturalized account of semantics reveals an underlying system This perspective suggests that language, as a rule-governed activity, possesses a crucial normative aspect that cannot be fully explained through naturalistic means, a notion rooted in Wittgenstein's philosophy.
CHAPTER 1 CONCEPTS OF INFORMATION 32 reflections on meaning and rule-following (Wittgenstein, 1953).
Returning to quantum information theory, the following quotation from a recent article in Reviews of Modern Physics provides an apt illustration of the problematic claim that ‘Information is Physical’.
Quantum physics has the potential to significantly impact information and computation, fundamentally altering their foundational principles.
The realization that information possesses a physical nature marks a significant shift in our understanding, as highlighted by Landauer (1991; 1996; 1961) Information is stored on tangible media, cannot be transmitted faster than the speed of light in a vacuum, and adheres to the principles of natural laws This perspective emphasizes that information is not just a byproduct of physical devices, but rather a physical entity in its own right.
The laws governing information are inherently limited by the principles of physics, especially those of quantum physics, as noted by Galindo and Martín-Delgado (2002).
Whilst illustrating the problem, this passage also invites a simple response, one indicat- ing the lines of a solution.
Let’s pick out three phrases:
1 ‘The statement that information is physical does not simply mean that a computer is a physical object’
2 ‘in addition information itself is a physical entity’
3 ‘In turn, this implies that the laws of information are restricted or governed by the laws of physics.’
Statement (2) claims to offer a new ontological perspective that is either influenced by or influences quantum information theory The challenge lies in grasping the true meaning of this impressive phrase and, importantly, determining the role it is intended to fulfill.
Statement (3), which interprets 'laws of information' as the principles governing information processing, is crucial for understanding the potential of information processing in physical systems, particularly in the context of quantum mechanics.